The WikibaseMigrator, also known as the Beam-me-up tool, automates the complex process of transferring data from Wikidata to FactGrid. Traditional methods such as manual creation or imports through QuickStatements and OpenRefine often prove to be time-consuming and error-prone. This new tool simplifies the entire process by automatically mapping properties and items between the two Wikibase instances, having already performed over 18,000 successful edits.

How to Migrate Entities

The migration process requires only Wikidata IDs as input and offers three flexible input methods: you can enter a single ID, provide a comma-separated list of IDs (make sure you do not end with a comma), or use a SPARQL query. After entering the IDs in the input field, a preview table displays the selected entities with their English labels, allowing you to verify your selection before proceeding.

When you click “Run matching!”, the tool queries the selected entities’ data from Wikidata and begins the translation process. It extracts all properties and items from the dataset and searches for corresponding mappings in FactGrid. Using these mappings, the tool translates the entities to FactGrid entities. If an entity already exists in FactGrid, the tool will augment it with the new statements from Wikidata. This process may take several seconds, depending on the number of entities selected and the complexity of their statements.

Once the translation is complete, you will see the results for each entity, indicating whether the process will create a new entry or augment existing ones. At this stage, you can add a research project ID (P131) if the entities belong to your research project, which adds a corresponding statement to each migrated entity. As with any wiki edit, you can provide a summary explaining the reason for the migration—this is particularly recommended for large imports.

The Migration Process

After reviewing the translation and clicking Beam me up!, the actual migration begins. A progress bar keeps you informed of the process. Upon completion, you receive a comprehensive overview table of the created or augmented entities, which you can download for your records or further additions of new statements you want to make. This table includes both the Wikidata and FactGrid IDs, along with detailed migration information.

The tool handles property type mismatches between Wikidata and FactGrid through automatic type casting where possible. For instance, it can convert string values to quantities and manage monolingual text to string conversions and vice versa. Any such transformations are documented in the migration details column of the results table.

If a migration fails, the tool provides a separate table showing the affected Wikidata ID and the reason for the failure. During testing, most failures were related to existing sitelinks, as Wikidata supports defining sitelinks to redirects, which FactGrid does not.

 

Entity Augmentation and Advanced Features

The tool employs a sophisticated approach to augmenting existing entities, optimized for multiple augmentation cycles. Statements are considered equal if their main values match and either has no qualifiers or their qualifier sets are identical or one qualifier set is empty. In such cases, the tool merges references and qualifiers intelligently, preventing duplicate statements that might occur with the default merging strategy.

This feature proves particularly valuable when migrating interconnected data, such as family relationships. For example, when migrating a group of related persons with Mother (P142), Father (P141) and Child (P150) relationships, the tool can handle the circular dependencies effectively through multiple migration passes.

For example when migrating the list of Q81642270, Q81642507, Q28085 (assuming that they are not already in FactGrid) the statements with property mother (P25) and father (P22) will not be migrated as the target entities do not exist at the time of translation and thus do not have a FactGrid ID yet. But applying the migration a second time to the same list of entities leads to the augmentation of the now existing FactGrid entities allowing to migrate the statements with the properties mother (P25) and father (P22).

 

Technical Implementation and Additional Tools

All migrations are performed under the user’s account credentials, with each edit tagged to indicate it was made using the tool.

The Wikidata ID of the original entity is automatically added as a sitelink to newly created entities, ensuring proper linking between the two databases and facilitating future augmentations. It should be noted that the back reference is also configurable and can also be configured as external id with a property which would be a cleaner solution.

To complement the WikibaseMigrator, a Wikidata bot called FactGridSync periodically queries FactGrid’s latest edits and updates the corresponding Wikidata entries with FactGrid IDs. This synchronization covers all FactGrid entities, not just those migrated using the Beam-me-up tool.

 

For those interested in enhanced Wikidata functionality, I implemented the UI extension FactGridLinker, initially for debugging, that simplifies checking whether Wikidata entities exist in FactGrid. This user script can be enabled through your commons.js configuration (see here for details).

The WikibaseMigrator’s versatility extends beyond the Wikidata→FactGrid

relationship—it can be configured to work between any Wikibase instances with proper configuration.

It was successfully used for subsetting the CEUR-WS data into its own wikibase instance with just a few queries that defined entities of the subset and was performed in under two hours.

Should you encounter any issues while using the tool, you can report them on the my talk page or by opening a issue on GitHub.

---

Image Maniere universelle de M. Desargues, pour pratiquer la perspective par petit-pied… Planche 4 Gallica France