CDCAGG Client

This package implements CESSDA Metadata Aggregator client.

sync.py

Synchronize records to CESSDA Metadata Aggregator DocStore.

This module provides an entry point used for syncronizing records found from XML files to the DocStore.

Synchronization dedulicates records based on provenance information. If two records share the same baseUrl + identifier combination in one of their provenance items, they are considered to be the same record. In that case the new record always overwrites the old one.

Deduplication is implemented in StudyMethods.

class cdcagg_client.sync.StudyMethods(cache)

Implement StudyMethods subclass of CollectionMethods

Implement methods query_record() and query_distinct_ids() that are abstract in base class. Override method update_record() to correctly handle provenance info.

async query_distinct_ids()

Query distinct IDs from collection that are not deleted.

Returns:

Distinct ids

Return type:

set

async query_record(record)

Query record from Document Store.

This method is called from upsert() method. If this method returns None, then the upsert() will never call update_record(), but will call create_record() instead.

This query uses elemMatch to look for the identifier and base_url from within the same provenance item. It loops throught all provenance items of record to query for a record with matching provenance base_url + identifier combination. If a match is found, it is returned.

Parameters:

record (cdcagg_common.records.Study) – Study to query for.

Returns:

Result of the query.

Return type:

Instance of Study or None.

async update_record(new, old)

Update existing Document Store record.

Override kuha_client.CollectionMethods.update_record() to handle provenance data correctly.

Parameters:

• new (cdcagg_common.records.Study) – New record.

• old (cdcagg_common.records.Study) – Old record.

Returns:

False if record does not need updating.

Return type:

bool

cdcagg_client.sync.cli()

Start the program from command line.

Load configuration, run program, log and re-raise propagated exceptions.

cdcagg_client.sync.configure()

Declare configuration options and load settings.

Returns:

Loaded settings.

Return type:

argparse.Namespace

cdcagg_client.sync.run(settings)

Run the program with ‘settings’.

Load BatchProcessor and call BatchProcessor.upsert_run()

:param argparse.Namespace settings: Use settings to run the program.