macros to use flags

Add code to bdb_attribute and bdb_group where use TXN id and to
provide error, but need to rework callers (and their callers) to
ensure error is properly bubbled up to the backend operation routine
handling the transaction.  Ugh.

rework them slightly to avoid computations which might result in underflow.
Rename them for consistency with other berval routines.
Remove some utf8 lint.