Database Modules

hedwig.db.compat

hedwig.db.compat.scalar_subquery(select)

In newer SQLAlchemy apply the scalar_subquery method of a selectable.

This implementation does nothing, for versions pre-dating the introduction of this method.

hedwig.db.compat.row_as_dict(row)

Convert a row to a dictionary which can be modified.

Constructs the dictionary using the row’s items method.

hedwig.db.compat.row_as_mapping(row)

Convert the row to a form suitable for use as function **kwargs.

This implementation does nothing, as the row itself can be used.

hedwig.db.control

class hedwig.db.control.RecordUpdate(id, value, updates, value_unique_key, previous_unique_key, deferred)

_asdict()

Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('id', 'value', 'updates', 'value_unique_key', 'previous_unique_key', 'deferred')

classmethod _make(iterable)

Make a new RecordUpdate object from a sequence or iterable

_replace(**kwds)

Return a new RecordUpdate object replacing specified fields with new values

deferred

Alias for field number 5

id

Alias for field number 0

previous_unique_key

Alias for field number 4

updates

Alias for field number 2

value

Alias for field number 1

value_unique_key

Alias for field number 3

class hedwig.db.control.Database(engine, query_block_size=50)

_mem_ctr = count(0)

_transaction(_conn=None)

Private context manager method for handling database transactions.

Obtains a lock and then yields a connection object. SQLAlchemy errors are trapped and re-raised as our DatabaseError, other than for IntegrityError which is re-raised as DatabaseIntegrityError.

If “_conn” is not None, however, simply yields its value. This is so that methods can optionally take a transaction argument for when they are called from within an existing transaction.

_exists_id(conn, table, id_)

Test whether an identifier exists in the given table.

_iter_stmt(stmt, iter_field, iter_list)

Generate sequence of query statements.

This method takes an SQLAlchemy statement and yields a number of statements with an additional where clause. This clause requires the field iter_field to have a value in iter_list. When iter_list is larger than the object’s query_block_size attribute then multiple statements will be returned, where the list is broken into blocks of this size.

If iter_field is None then the given stmt is yielded as-is.

Parameters:

• stmt – statement to be modified.

• iter_field – field being searched.

• iter_list – list (or other iterable) or search values.

_sync_records(conn, table, key_column, key_value, records, update_columns=None, verified_columns=(), forbid_add=False, forbid_delete=False, record_match_column=None, unique_columns=None, forbid_circular_reinsert=False)

Update a set of database records to match the given set of records.

Queries the given table for rows where the value of the specified key column matches the specified key value. Then compares the result to the specified collection of values and performs a series of inserts, updates and deletes to bring the database to a state matching the specified values. Records are matched by their “id” values, but the dictionary keys of “records” don’t matter. New records can be given with an “id” of None, or an “id” which doesn’t match an existing record.

If update columns are specified, then only those columns are considered, both for comparison and updating.

key_column and update_columns should be a column object and a list of column objects respectively. key_column and key_value can alternatively be tuples/lists (of equal length) for cases where multiple columns are needed to determine membership of the set of rows being updated.

If a column is being updated which is listed in “verified columns” then a column called “verified” is set to False. (If verified columns exist, then update_columns should probably be set to prevent the “verified” column being edited!)

If “record_match_column” is specified then it will determine the column to use to match existing records to those given. If not specified then the table’s “id” column will be used. Note this only applies to the existing database records – we still expect to use the “id” field of the given records.

The “unique_columns” argument should be used to indicate which columns being updated have unique constraints in the database. If a set of circular updates involving the unique columns is detected, then the loop will be broken by deleting and re-inserting one of the records, unless the “forbid_circular_reinsert” argument is given, in which case an error is raised. Be aware that re-insertion currently only retains the columns being updated! If a syncing operation is added for which this is a problem, the insertions should be split into new records and re-insertions so that they can be handled separately.

_remove_orphan_records(conn, table, column_link)

Remove entries from table which do not have a reference in column_link to their id field.

Returns:

the number of rows deleted

hedwig.db.engine

hedwig.db.engine.engine_connect(dbapi_connection, connection_record)

Listener for engine connections.

• Enable foreign key constraints if the engine is SQLite.

• Record the associated PID.

hedwig.db.engine.engine_checkout(dbapi_connection, connection_record, connection_proxy)

Listener for engine check-outs.

• Checks that we have the same PID as we did at connection time.

(Based on the example in the “Connection Pooling” section of the SQLAlchemy documentation.)

hedwig.db.engine.pool_checkout(dbapi_connection, connection_record, connection_proxy)

Listener for pool check-outs.

• Checks that the connection is alive.

(Based on the pessimistic example in the “Connection Pooling” section of the SQLAlchemy documentation.)

hedwig.db.engine.get_engine(url, **kwargs)

Creates an SQLAchemmy database engine object for the given URL.

hedwig.db.meta

hedwig.db.meta._note_cols()

hedwig.db.meta._calculation_cols()

hedwig.db.meta._oauth_cols()

hedwig.db.meta._fig_cols()

hedwig.db.meta._fig_preview_cols()

hedwig.db.meta._fig_thumbnail_cols()

hedwig.db.meta._request_cols()

hedwig.db.type

class hedwig.db.type.AutoEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

JSON encoder class which will use object helper methods to convert them to a serializable form.

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return super().default(o)

class hedwig.db.type.JSONEncoded(*args: Any, **kwargs: Any)

Represents a data structure as JSON in a unicode text column.

cache_ok = False

process_bind_param(value, dialect)

process_result_value(value, dialect)

hedwig.db.util

hedwig.db.util.memoized(f)

Decorator to cache database metehod results.

Expects the cache dictionary and database object to be provided as the first two arguments. The database object is then passed as the first argument to the decorated function. If the cache object is None, no memoization is performed.

hedwig.db.util.require_not_none(f)

Decorator which checks that the return value of a function is not none. If it is, then NoSuchRecord is raised.

class hedwig.db.util.ReadOnlyWrapper(db)

Wrapper to restrict access to a database controller object to only read-only methods.

Read-only methods are considered to be those starting get_ or search_.

If a technique to provide real read-only access to a database via SQLAlchemy emerges then it should be used instead of this! This wrapper is not foolproof and shouldn’t be used for security. It just aims to provide some safeguard against accidental writes to the database, such as when testing operations in dry-run mode.