timelink.mhk.models package

Handling of Database persistence for the Person Oriented Model (POM).

The classes on this module represent the core entities in the POM: entities,sources, acts, people, objects, attributes and relations. SQLAlchemy is used as the underlying ORM library.

They allow storage and query and also dynamic mapping of new types of entities defined in kleio source files, through the PomSomMapper class.

For a detailed description of Timelink data models see: Timelink data concepts and models.

Also check timelink.mhk.models.pom_som_mapper.PomSomMapper

(c) Joaquim Carvalho 2021. MIT License, no warranties.

Submodules

timelink.mhk.models.act module

Implementation of Acts in POM model (c) Joaquim Carvalho 2021. MIT License, no warranties.

class timelink.mhk.models.act.Act(**kwargs)

Represents an Act, i.e. a record of some event in a historical document.

Examples of acts are: baptisms, marriages, purchase deeds, wills, court records.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

a string uniquely identifying the act

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

loc

location of the act, eg church, notary office

Type:

str

obs

any observations or comments.

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

ref

archival reference

Type:

str

the_date

the date of the act in Kleio format AAAAMMDD

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

the_type

the type of the act

Type:

str

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.attribute module

class timelink.mhk.models.attribute.Attribute(**kwargs)

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.base module

timelink.mhk.models.base_class module

class timelink.mhk.models.base_class.Base(**kwargs: Any)

Base class, all modules must import this to share Metadata

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

metadata: ClassVar[MetaData] = MetaData()

Refers to the _schema.MetaData collection that will be used for new _schema.Table objects.

See also

orm_declarative_metadata

registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>

Refers to the _orm.registry in use where new _orm.Mapper objects will be associated.

timelink.mhk.models.base_mappings module

timelink.mhk.models.db module

timelink.mhk.models.db.TimelinkDB

alias of TimelinkMHK

class timelink.mhk.models.db.TimelinkMHK(conn_string=None, sql_echo=False)

Provide access to a Timelink-MHK database

Example

from timelink.mhk.model.db import TimelinkMHK

dbsys = TimelinkMHK(“sql alchemy connection string”)

For the interaction of “engine” and “session” alternative configurations see https://docs.sqlalchemy.org/en/14/orm/session_basics.html

Parameters:

conn_string – a sqlalchemy connection URL, str.

create_tables()

Creates the tables from the current ORM metadata if needed

Returns:

None

drop_db(session)

This will drop all timelink related tables from the database. It will not touch non timelink tables that might exist. :param session: :return:

ensure_all_mappings(session)

Ensure that all database classes have a table and ORM class

get_engine() → None

Return sqlalchemy engine

get_metadata() → MetaData

Return sqlalchemy metada

get_session() → sessionmaker

Return sqlalchemy session

init_db(conn_string, sql_echo=False)

Create an engine from a connection string. Check if database is properly setup up by creating core tables, PomSomMappers and ORM classes. Database should be ready for import after this method is called.

Parameters:

conn_string – SQLAlchemy connection string

Returns:

load_database_classes(session)

Populates database with core Database classes :param session: :return:

sa_engine() → None

DEPRECATED use TimelinkDB.get_engine()

table_names()

Current tables in the current database

timelink.mhk.models.entity module

class timelink.mhk.models.entity.Entity(**kwargs)

ORM Model root of the object hierarchy.

All entities in a Timelink/MHK database have an entry in this table. Each entity is associated with a class that allow access to a specialization table with more columns for that class.

This corresponds to the model described as “Joined Table Inheritance” in sqlalchemy (see https://docs.sqlalchemy.org/en/14/orm/inheritance.html)

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

classmethod get_entity(id: str, session=None)

Get an Entity from the database. The object returned will be of the ORM class defined by mappings. :param id: id of the entity :param session: current session :return: an Entity object of the proper class for the mapping

classmethod get_orm_entities_classes()

Currently defined ORM classes that extend Entity

(including Entity itself)

Returns:

List of ORM classes

Return type:

list

classmethod get_orm_for_pom_class("act")

will return the ORM class corresponding to the pom_class “act”

classmethod get_orm_for_table("acts")

will return the ORM class handling the “acts” table

classmethod get_som_mapper_ids()

Ids of SomPomMapper references by orm classes

Returns:

List of strings

Return type:

List[str]

classmethod get_som_mapper_to_orm_as_dict()

Return a dict with pom_class id as key and ORM class as value

classmethod get_subclasses()

Get the subclasses of Entity

classmethod get_tables_to_orm_as_dict()

Return a dict with table name as key and ORM class as value

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.object module

(c) Joaquim Carvalho 2021. MIT License, no warranties.

class timelink.mhk.models.object.Object(**kwargs)

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.person module

class timelink.mhk.models.person.Person(**kwargs)

Represents a person.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.pom_som_mapper module

class timelink.mhk.models.pom_som_mapper.PomClassAttributes(**kwargs)

Attribute of a PomClass. Maps kleio group element to table columns

the_class: id of the PomSomClass this attribute is attached to. name : name of of the attribute colname : name of the column in the corresponding table colclass : class of the column (element source if different from colname) coltype : type of the column colsize : size of the column, int colprecision: precision of the column (if decimal), int pkey : if > 0 order of this column in the primary key of the table

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

class timelink.mhk.models.pom_som_mapper.PomSomMapper(**kwargs)

Represents a mapping between a Kleio Group in the Source Oriented Model (Som) and a relational database entity in the Person Oriented Model (Pom). This class corresponds to the table “classes” in a Timelink-MHK database, and the associated “class_attributes” table. Together the two tables describe a Som-Pom mapping, that define how Kleio groups are stored in the relational database.

This class can generate tables and ORM objects that can store a Kleio Group in the database.

Fields

Id:

name of this PomSomMapper, singular form

Table_name:

name of the table in Pom, plural form

Group_name:

name of Som group that maps to this table

Super_class:

name of PomSom class extended by this one

For the core kleio groups (source,act,person,object, relation,attribute) the mapping information is predefined at database creation time and the ‘classes’ and ‘class_attributes’ populated accordingly.

The Kleio translator can provide new mappings for new groups that are created for specific sources. The mapping information between new groups and the database is embedded in the translator output of new sources in the form of data for the tables mapped to the PomSomMapper.

For a mapping between a Som Group and a Pom table to be fully operational it is necessary that:

1. The tables “classes” and “class_attributes” contain the mapping information, normally populated during the initialization process of the database (DBSystem.db_init) and updated during import, as new sources define new mappings dynamically.

2. A table for storing the elements of the Som Group. This is either a basic core table (persons,objects,acts,sources,…) or a table that extends a basic core table with extra columns.

3. An ORM sqlalchemy model that represents the Pom model and joins the information of the inheritance hierarchy, by mapping to the necessary tables.

The name of the table for a given group and the correspondence between group elements and table columns is handled by the PomSomMapper. If the group adds extra information then a new table is created that “extends” an existing core table, by what is called a “joined inheritance hierarchy” (see https://docs.sqlalchemy.org/en/14/orm/inheritance.html)

To ensure that all the three dimensions exist in a given context it is necessary that:

1. When creating a new database, the core Pom tables should be created using sqlalchemy metadata.create_all(bind=engine). Then the tables “classes” and “class_attributes” must be populated with the mapping of the core Som and Pom groups and entities.

See timelink.models.base_mappings.py for the core data for bootstrapping of the mapping information.

Both steps are ensured by DBSystem.init_db

2. When importing new sources, if a new mapping was generated at the translation step, it is necessary first to populate “classes” and “class_attributes” with the mapping information, included in the translation source, then generate the table for the group information (if it does not exist) and the ORM mapping. This step is done by the method PomSomMapper.ensure_mapping(session) which must be called during the import process.

3. When connecting to a database created by a legacy version of MHK, it is necessary to ensure that all the ORM mappings are created, by examining the information in the “classes” table and checking if the ORM mapping and table representations exist. This is done by the PomClass.ensure_all_mappings() method.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

column_to_class_attribute(colname: str, session: None) → PomClassAttributes

Return the class attribute corresponding to a column name.

Parameters:

colname (str) – The name of a column in the mapped table.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

element_class_to_column(eclass: str, session: None) → str

Return the column name corresponding to a group element class.

Parameters:

eclass (str) – The class of an element (included in the export file).

Returns:

The name of the column corresponding to this element in the mapped table.

Return type:

str

classmethod ensure_all_mappings(session)

Ensures that every class currently defined in the database has a python table object and a python ORM object

Returns:

None

ensure_mapping(session=None)

Ensure that a table exists to support this SOM Mapping and an ORM class is created to represent data of objects from this mapping.

Checks if there is a table definition to support this mapping. If not a new table definition is created with information from the class attributes.

A new ORM class is also created for mapping the new table. The new ORM class will extend the superclass ORM mapping

classmethod get_orm_for_group(groupname: str)

PomSomMapper.get_orm_for_groupname(“act”)

will return the ORM class corresponding to the groupname “act”

classmethod get_pom_class(pom_class_id: String, session)

Return the pom_class object for a given pom_class_id.

See also Entity.get_orm_for_pom_class

Parameters:

pom_class_id – the id of a pom_class

classmethod get_pom_class_from_group(group: KGroup, session=None)

Returns the PomSomMapper for a given group

classmethod get_pom_class_ids(session)

Return all the pom_som_class ids as a list :return:

classmethod get_pom_classes(session) → List[PomSomMapper] | None

Get the pom_classes from database data.

Note that this method does not ensure that a ORM mapper is created for each class. Use pom_class.ensure_mapping() method to dynamically produce the ORM class

Returns:

A list of SomPomMappers object for each class in the db

classmethod get_pom_id_by_group_name(session, kname)

Returns the PomSomMapper id for a given group name

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

classmethod kgroup_to_entity(group: KGroup, session=None, with_pom=None) → Entity

Store a Kleio Group in the database.

Parameters:

• group – a Kleio Group

• with_pom – id of a PomSom class to handle storing this group

Returns:

An ORM object that can store this group in the database

To produce the ORM-POM representation of a group we need to find the PomSomMapper specific to that group, using the following:

• if with_pom is given with a PomSomMapper id we fetch that

• if the group was imported it should have _pom_class_id

• if neither then we search for a PomSom mapper with “groupname”

  equal to the name of the group.

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

classmethod store_KGroup(group: KGroup, session=None)

Store a Kleio Group in the database

Will recursively store all the groups included in this group.

If the group is already in the database it will be deleted, as well as all included groups.

This is the main method that import Kleio transcripts into the database.

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.relation module

class timelink.mhk.models.relation.Relation(**kwargs)

Represents a relation between two entities.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.source module

(c) Joaquim Carvalho 2021. MIT License, no warranties.

class timelink.mhk.models.source.Source(**kwargs)

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

contains

list of Entity objects contained in this entity

Type:

list(Entity)

groupname

name of the kleio group that produced this entity

Type:

str

id: Any

unique identifier for the entity

Type:

str

indexed

when this entity was added to the full text index

Type:

datetime

inside

id of the entity inside which this occurred.

Type:

str

pom_class

name of the class. Links to pom_som_mapper class

Type:

str

the_level

the nesting level of this entity in the source

Type:

int

the_line

line in which the entity occurred in the source

Type:

int

the_order

sequential order of this entity in the source

Type:

int

updated

when this entity was updated in the database

Type:

datetime

timelink.mhk.models.system module

Models for system tables

Tables defined here:

• syspar: system parameters

• syslog: system log

class timelink.mhk.models.system.KleioImportedFile(**kwargs)

Represents a Kleio file imported in the database

Fields:

path: path of the file name: name of the file structure: structure name translator: translator name translation_date: date of translation nerrors: number of errors nwarnings: number of warnings error_rpt: error report warning_rpt: warning report

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

class timelink.mhk.models.system.LogLevel(value)

Log level

Fields:

debug: debug info: info warning: warning error: error critical: critical

class timelink.mhk.models.system.ParTypeSchema(value)

Parameter type

Fields:

string: string integer: integer float: float date: date boolean: boolean list: list

class timelink.mhk.models.system.SysLog(**kwargs)

System log table

Fields:

seq: sequence number time: time of log entry level: log level origin: origin of log entry message: log message

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

class timelink.mhk.models.system.SysLogCreateSchema(*, level: LogLevel, origin: str, message: str)

System log create in the Timelink app

Fields:

level: log level origin: origin of log entry message: log message

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

model_config: ClassVar[ConfigDict] = {'from_attributes': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class timelink.mhk.models.system.SysLogSchema(*, seq: int, time: datetime, level: LogLevel, origin: str, message: str)

System log in the Timelink app

Fields:

seq: sequence number time: time of log entry level: log level origin: origin of log entry message: log message

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

model_config: ClassVar[ConfigDict] = {'from_attributes': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class timelink.mhk.models.system.SysPar(**kwargs)

System parameters table

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

class timelink.mhk.models.system.SysParSchema(*, pname: str, pvalue: str, ptype: ParTypeSchema, obs: str)

System parameters in the Timelink app

Fields:

pname: parameter name pvalue: parameter value ptype: parameter type obs: parameter description

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

model_config: ClassVar[ConfigDict] = {'from_attributes': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].