DrupalCon is always a great opportunity to share experiences, knowledge and some funny stories about our daily work.
What caught my attention last year was the reaction from fellow drupalers when I said I was using the Migration API to move content from Drupal 7 to Drupal 8. That made me feel that people should get more comfortable with this API (it doesn’t bite!) to take advantage of their features/benefits, rather than using custom data import methods.
Of course, there’s always a considerable learning curve around coding in Drupal 8. We all know about the changes / improvements between D8 and previous versions (I swear that moving stuff from Drupal 8 to 9 will be waaaay easier than this), so figuring migration out completely actually took me a few months.
Migration API is based on the following principles:
There is a source plugin where you perform the data extraction from a given resource (can be a Drupal < 9 site, a CSV file, a RSS input or an XML)
The process plugins allow you to alter or adjust the source data to compile your final destination’s specs
The destination plugin connects the resulting data with your brand-new Drupal site
Drupal offers someextremely useful plugins (source, process, destination) out-of-the-box that I suggest you take a look at:
Here are a few of the most common issues I experienced using the Migrate API in Drupal 8:
Moving nodes with referenced terms
Moving media elements (images/audio/videos)
Referenced nodes/terms
A migration is a procedure/job, defined by a YAML file. These migration files are usually located in modules implementing the Migrate API, in the following path (for our module called “tbf_migrate”):
tbf_migrate/src/config/install/migrate_plus.migration.agency_terms.yml
This file provides all the instructions for the migration to be executed:
# Agency terms from D7
id: agency_terms
label: Agency Terms
migration_group: d7_content_group
source:
plugin: term_content
vocabulary_machine_name: agencies
process:
vid:
plugin: default_value
default_value: agencies
name: name
description: description
weight: weight
changed: timestamp
destination:
plugin: entity:taxonomy_term
migration_dependencies:
In order to understand migrations and the proper way to implement them, we must go from specific to general. This means, first moving roles and users, then taxonomy content, media (and files), then nodes that are referenced by other nodes, then more complex nodes, referencing terms, media and other nodes and entities, and finally URLs (I understand there are many ways to do this last thing).
First of all, is recommended to add some required dependencies to your custom module. Here is an example of how the [module].info.yml might look.
tbf_migrate.info.yml:
name: TBF Migrate Helper
description: Performs Migration from D7 to D8.
package: Other
dependencies:
– migrate
– migrate_plus
– migrate_tools
type: module
core: 8.x
As mentioned before, Migrations start with a source. For our implementation, we required the Drupal 7 database from the old site, and we add it as a secondary database to our settings.php file:
// Adding migrate database to handle migrations
$databases[‘migrate’][‘default’] = array(
‘driver’ => ‘mysql’,
‘host’ => ‘old-site.com’,
‘username’ => ‘migrator_user’,
‘password’ => ‘xxxxxxxxxxxxxx’,
‘database’ => ‘drupal_7_db’,
‘prefix’ => ”,
);
Depending on where you are running your migration (local or dev environment), you will need to make sure that you can access the data source.
After setting the database configuration, we need to create a Migration Group, in our case we placed all migration files within our module folder:
tbf_migrate/config/install
Our migration group YAML file is called migrate_plus.migration_group.d7_content_group.yml (the migrate_plus prefix on every migration or migration_group file is required).
#id of the config
id: d7_content_group
label: Content Migration Group
description: Drupal 7 migrations group.
source_type: Drupal 7
# In this section we set a database connection defined in settings.php
# So all migrations in this group will use this source.
shared_configuration:
source:
key: migrate
# Here we set a dependency on the module itself.
# This is necessary setting that deletes configs from a database
# on uninstallation of the module.
dependencies:
enforced:
module:
– tbf_migrate
You can create more migration groups by adding more YAML files with the specified structure.
Along with already packaged Migration API plugins, I suggest you review all the modules you have installed on your Drupal 8/9 site, because many of them have Migration API support and migration files that can be useful.
And this is the case with the built-in User module. It offers some migrations out-of-the-box, and I used two of them:
d7_user_roles
d7_user
And this is how we start migrating user info from the Drupal 7 database within our Drupal 8 folder using Drush:
drush migrate:import d7_user_roles
drush migrate:import d7_user
After running these commands, drush will return the status for the performed migration (imported/ignored row amounts).
Note: It is going to be easier to migrate a Drupal 7 site to Drupal 8. So if you are on Drupal 6, I would suggest first upgrading to Drupal 7. Likewise, before moving to Drupal 9, you will first want to upgrade to Drupal 8.
For this one, I created my own migration plugins based on the existing d7_node source migration plugin. Then under the fields area I separated fields based on type (single fields, reference fields, media fields).
Here’s a sample migration for a really simple node type with title and body.
id: d7_basic_page_content
label: Basic Page Content
migration_group: d7_content_group
audit: true
migration_tags:
– Drupal 7
– Content
source:
plugin: tbf_node_generic_content
node_type: ‘page’
process:
langcode:
plugin: default_value
source: language
default_value: “und”
title: title
uid: node_uid
status: status
created: created
changed: changed
promote: promote
sticky: sticky
revision_uid: revision_uid
revision_log: log
revision_timestamp: timestamp
type:
plugin: content_type_mapper
default_value: type
‘body/format’:
plugin: default_value
default_value: ‘full_html’
‘body/value’: body_value
‘body/summary’: body_summary
destination:
plugin: entity:node
bundle: basic_page
migration_dependencies:
required:
– d7_user
Let’s break this big YAML down to analyze every aspect of this migration:
id, label, migration_group: the YAML file opens with these required properties
source: for the source we created our own source plugin, and we passed a parameter called node_type for the plugin to filter only Basic Page contents. Migration source plugins are PHP classes previously created and stored in /tbf_migrate/src/Plugin/migrate/source/. We’ll talk about this more later.
process: in this part of the migration, we prepare all the information before storing it in our new Drupal 8 site. Here you can perform mappings, parsing, replacements, providing fixed values. As mentioned previously, Drupal offers a nice plugin set for handling the most common processing situations. In this case, the processing is simple because we are mapping most of the values to their destination, except for the langcode and body/format parts. Here we used one of the most common processing plugins called default_value to explicitly assign a value. Since our website doesn’t contain multiple languages, we set the langcode to be ‘und’ by default. Same for body/format, where its default is going to be ‘full_html’. Please notice there is another plugin, for the type entry. Here’s our first custom plugin called content_type_mapper, and we will share the details later.
migration_dependencies: Since the D8 destination had their content types created we didn’t need to use the d7_node_type migration job from the Node module. In this case, we kept the d7_user dependency because we need users migrated before any content migration for authoring purposes.
Sometimes when the included migration plugins don’t fill your requirements, creating custom plugins is necessary. As mentioned many times here, you can pick an existing plugin and customize the code within your module. That’s what happened with my first sample called tbf_node_generic_content. For this plugin I took the code from the Node module (web/core/modules/node/src/Plugin/migrate/source/d7/Node.php), and copied it into the custom module folder we created (web/modules/custom/tbf_migrate/src/Plugin/migrate/source/d7/NodeGenericContent.php).
Note: This is a large chunk of code that I included for your reference. You can go through it and figure out what might be useful for your use case.
<?php
namespace Drupal\tbf_migrate\Plugin\migrate\source\d7;
use Drupal\Core\Extension\ModuleHandlerInterface;
use Drupal\migrate\Row;
use Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity;
use Drupal\Core\Database\Query\SelectInterface;
use Drupal\Core\Entity\EntityManagerInterface;
use Drupal\Core\Extension\ModuleHandler;
use Drupal\Core\State\StateInterface;
use Drupal\migrate\Plugin\MigrationInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
/**
* Drupal 7 node source from database.
*
* @MigrateSource(
* id = “tbf_node_generic_content”,
* source_module = “node”
* )
*/
class NodeGenericContent extends FieldableEntity {
/**
* The module handler.
*
* @var \Drupal\Core\Extension\ModuleHandlerInterface
*/
protected $moduleHandler;
/**
* {@inheritdoc}
*/
public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, StateInterface $state, EntityManagerInterface $entity_manager, ModuleHandlerInterface $module_handler) {
parent::__construct($configuration, $plugin_id, $plugin_definition, $migration, $state, $entity_manager);
$this->moduleHandler = $module_handler;
}
/**
* {@inheritdoc}
*/
public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) {
return new static(
$configuration,
$plugin_id,
$plugin_definition,
$migration,
$container->get(‘state’),
$container->get(‘entity.manager’),
$container->get(‘module_handler’)
);
}
/**
* The join options between the node and the node_revisions table.
*/
const JOIN = ‘n.vid = nr.vid’;
/**
* {@inheritdoc}
*/
public function query() {
// Select node in its last revision.
$query = $this->select(‘node_revision’, ‘nr’)
->distinct(true)
->fields(‘n’, [
‘nid’,
‘type’,
‘language’,
‘status’,
‘created’,
‘changed’,
‘comment’,
‘promote’,
‘sticky’,
‘tnid’,
‘translate’,
])
->fields(‘nr’, [
‘vid’,
‘title’,
‘log’,
‘timestamp’,
]);
$query->addField(‘n’, ‘uid’, ‘node_uid’);
$query->addField(‘nr’, ‘uid’, ‘revision_uid’);
$query->innerJoin(‘node’, ‘n’, static::JOIN);
// If the content_translation module is enabled, get the source langcode
// to fill the content_translation_source field.
if ($this->moduleHandler->moduleExists(‘content_translation’)) {
$query->leftJoin(‘node’, ‘nt’, ‘n.tnid = nt.nid’);
$query->addField(‘nt’, ‘language’, ‘source_langcode’);
}
$this->handleTranslations($query);
// Add user email to match UID to the current Drupal created ones
$query->leftJoin(‘users’, ‘u’, ‘u.uid = n.uid’);
$query->addField(‘u’, ‘mail’, ‘author’);
if (isset($this->configuration[‘node_type’])) {
$query->condition(‘n.type’, $this->configuration[‘node_type’]);
}
if (isset($this->configuration[‘bundles’]) && count($this->configuration[‘bundles’])) {
$query->condition(‘n.type’, $this->configuration[‘bundles’], ‘IN’);
}
// Add content type label
$query->leftJoin(‘node_type’, ‘nt’, ‘n.type = nt.type’);
$query->addField(‘nt’, ‘name’, ‘type_label’);
$query->leftJoin(‘field_revision_body’, ‘nb’, ‘nb.revision_id = n.vid’);
$query->addField(‘nb’, ‘body_value’, ‘body_value’);
$query->addField(‘nb’, ‘body_summary’, ‘body_summary’);
// We need to set some key fields in the groupBy function
$group_concat_used = false;
$groupBy = [];
$groupBy = [‘n.nid’, ‘nb.body_value’, ‘nb.body_summary’];
// Adding fields from source config
// Basic fields – Fields containing values in revision table.
if (isset($this->configuration[‘basic_fields’])) {
$i = 0;
foreach ($this->configuration[‘basic_fields’] as $field) {
$query->leftJoin(‘field_revision_’.$field, ‘f’.$i, ‘f’.$i.’.revision_id = n.vid’);
$query->addField(‘f’.$i, $field.’_value’, $field);
$groupBy[] = ‘f’.$i.’.’.$field.’_value’;
$i++;
}
}
if (isset($this->configuration[‘taxonomy_fields’])) {
$i = 0;
foreach ($this->configuration[‘taxonomy_fields’] as $field) {
$query->leftJoin(‘field_revision_’.$field, ‘t’.$i, ‘t’.$i.’.revision_id = n.vid’);
$query->addExpression(‘GROUP_CONCAT(DISTINCT(t’.$i.’.’.$field.’_tid))’, $field.’_tid’);
$group_concat_used = true;
$i++;
}
}
if (isset($this->configuration[‘entity_fields’])) {
$i = 0;
foreach ($this->configuration[‘entity_fields’] as $field) {
$query->leftJoin(‘field_revision_’.$field, ‘e’.$i, ‘e’.$i.’.revision_id = n.vid’);
$query->addExpression(‘GROUP_CONCAT(DISTINCT(e’.$i.’.’.$field.’_target_id))’, $field.’_target_id’);
$group_concat_used = true;
$i++;
}
}
if (isset($this->configuration[‘media_fields’])) {
$i = 0;
foreach ($this->configuration[‘media_fields’] as $field) {
$query->leftJoin(‘field_revision_’.$field, ‘r’.$i, ‘r’.$i.’.revision_id = n.vid’);
$query->addField(‘r’.$i, $field.’_fid’, $field.’_fid’);
$groupBy[] = ‘r’.$i.’.’.$field.’_fid’;
$i++;
}
}
if ($group_concat_used || count($groupBy))
$query->groupBy(implode(‘, ‘, $groupBy));
return $query;
}
/**
* {@inheritdoc}
*/
public function prepareRow(Row $row) {
$nid = $row->getSourceProperty(‘nid’);
$vid = $row->getSourceProperty(‘vid’);
$type = $row->getSourceProperty(‘type’);
// If this entity was translated using Entity Translation, we need to get
// its source language to get the field values in the right language.
// The translations will be migrated by the d7_node_entity_translation
// migration.
$entity_translatable = $this->isEntityTranslatable(‘node’) && (int) $this->variableGet(‘language_content_type_’ . $type, 0) === 4;
$source_language = $this->getEntityTranslationSourceLanguage(‘node’, $nid);
$language = $entity_translatable && $source_language ? $source_language : $row->getSourceProperty(‘language’);
// Get Field API field values.
foreach ($this->getFields(‘node’, $type) as $field_name => $field) {
// Ensure we’re using the right language if the entity and the field are
// translatable.
$field_language = $entity_translatable && $field[‘translatable’] ? $language : NULL;
$row->setSourceProperty($field_name, $this->getFieldValues(‘node’, $field_name, $nid, $vid, $field_language));
}
// Make sure we always have a translation set.
if ($row->getSourceProperty(‘tnid’) == 0) {
$row->setSourceProperty(‘tnid’, $row->getSourceProperty(‘nid’));
}
// If the node title was replaced by a real field using the Drupal 7 Title
// module, use the field value instead of the node title.
if ($this->moduleExists(‘title’)) {
$title_field = $row->getSourceProperty(‘title_field’);
if (isset($title_field[0][‘value’])) {
$row->setSourceProperty(‘title’, $title_field[0][‘value’]);
}
}
// Point authoring info to the right User
$mail = $row->getSourceProperty(‘author’);
$uid = 1;
$users = \Drupal::entityTypeManager()->getStorage(‘user’)->loadByProperties([‘mail’ => $mail]);
$user = reset($users);
if ($user) {
$uid = $user->id();
}
$row->setSourceProperty(‘uid’, $uid);
return parent::prepareRow($row);
}
/**
* {@inheritdoc}
*/
public function fields() {
$fields = [
‘nid’ => $this->t(‘Node ID’),
‘type’ => $this->t(‘Type’),
‘type_label’ => $this->t(‘Type Label’),
‘title’ => $this->t(‘Title’),
‘body_value’ => $this->t(‘Body’),
‘body_summary’ => $this->t(‘Summary’),
‘node_uid’ => $this->t(‘Node authored by (uid)’),
‘revision_uid’ => $this->t(‘Revision authored by (uid)’),
‘created’ => $this->t(‘Created timestamp’),
‘changed’ => $this->t(‘Modified timestamp’),
‘status’ => $this->t(‘Published’),
‘promote’ => $this->t(‘Promoted to front page’),
‘sticky’ => $this->t(‘Sticky at top of lists’),
‘revision’ => $this->t(‘Create new revision’),
‘language’ => $this->t(‘Language (fr, en, …)’),
‘tnid’ => $this->t(‘The translation set id for this node’),
‘timestamp’ => $this->t(‘The timestamp the latest revision of this node was created.’),
];
// Basic fields – Fields containing values in revision table.
if (isset($this->configuration[‘basic_fields’])) {
foreach ($this->configuration[‘basic_fields’] as $field) {
$fields[$field] = $field;
}
}
if (isset($this->configuration[‘taxonomy_fields’])) {
foreach ($this->configuration[‘taxonomy_fields’] as $field) {
$fields[$field] = $field;
}
}
if (isset($this->configuration[‘entity_fields’])) {
foreach ($this->configuration[‘entity_fields’] as $field) {
$fields[$field] = $field;
}
}
if (isset($this->configuration[‘media_fields’])) {
foreach ($this->configuration[‘media_fields’] as $field) {
$fields[$field] = $field;
$fields[$field.’_fid’] = $field.’_fid’;
}
}
return $fields;
}
/**
* {@inheritdoc}
*/
public function getIds() {
$ids[‘nid’][‘type’] = ‘integer’;
$ids[‘nid’][‘alias’] = ‘n’;
return $ids;
}
/**
* Adapt our query for translations.
*
* @param \Drupal\Core\Database\Query\SelectInterface $query
* The generated query.
*/
protected function handleTranslations(SelectInterface $query) {
// Check whether or not we want translations.
if (empty($this->configuration[‘translations’])) {
// No translations: Yield untranslated nodes, or default translations.
$query->where(‘n.tnid = 0 OR n.tnid = n.nid’);
}
else {
// Translations: Yield only non-default translations.
$query->where(‘n.tnid <> 0 AND n.tnid <> n.nid’);
}
}
}
The addition to this plugin class was the field management code, this way we only use one migration to bring all the contents, term, media and node reference fields, and other field types.
The new D8 site needed to handle Content Types so that blog posts, press releases, letters and presentations would go into one general content type called “news”. For this reason, we created a process class to map content types to target the new machine names.
Process plugins go under the src/Plugin/migrate/process folder. Our sample is located in
web/modules/custom/tbf_migrate/src/Plugin/migrate/process/ContentTypeMapper.php
<?php
namespace Drupal\tbf_migrate\Plugin\migrate\process;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\Row;
use Drupal\migrate\Plugin\MigrateProcessInterface;
use \Drupal\Component\Utility\Html;
/**
* @MigrateProcessPlugin(
* id = “content_type_mapper”,
* )
*/
class ContentTypeMapper extends ProcessPluginBase implements MigrateProcessInterface {
/**
* {@inheritdoc}
*/
public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
$type = $row->getSourceProperty(‘type’);
switch ($type) {
case ‘page’:
$type = ‘basic_page’;
break;
case ‘blog’:
case ‘latest_news’:
case ‘presentation’:
case ‘press_release’:
$type = ‘blog’;
break;
case ‘litigation’:
$type = ‘tracker_litigation’;
break;
case ‘order_action’:
$type = ‘tracker_order_action’;
break;
default:
break;
}
return $type;
}
}
Now that we have migrated some simple entities, let’s go to the next level. There were some nodes in Drupal 8 that had media entities. In this case, we need to create some additional migrations to handle the conversion of Drupal 7 file/image fields to media entities in Drupal 8.
We had a “Profile” content type that contained a related image so we had to:
Move the files from the old D7 to the new D8’s files folder
Create a File migration job
Create a Media migration job
Then link the Media migration job to the Profile node migration job
The File migration YAML file is the following:
id: tbf_profile_images_files
label: Profile Image Files
migration_group: tbf_content
source:
plugin: tbf_media_generic_content
entity_type: node
bundle: profile
langcode: und
field_name: field_profile_image
destination:
plugin: entity:file
process:
uid:
plugin: default_value
default_value: 1
filename: file_name
file_path: file_path
uri:
plugin: file_copy
source:
– ‘file_path’
– ‘file_path’
file_exists: use existing
migration_dependencies:
And the Media migration YAML file looks similar:
id: tbf_profile_media_images
label: Profile Image Media
migration_group: tbf_content
source:
plugin: tbf_media_generic_content
entity_type: node
bundle: profile
langcode: und
field_name: field_profile_image
destination:
plugin: entity:media
process:
bundle:
plugin: default_value
default_value: image
name: title
uid:
plugin: default_value
default_value: 1
‘field_media_image/target_id’:
plugin: migration_lookup
migration: tbf_profile_images_files
source: fid
‘field_media_image/alt’: file_name
‘field_media_image/title’: file_name
migration_dependencies:
required:
– tbf_profile_images_files
The main difference between the previous migrations is that tbf_profile_images_files needs to be processed first, then must be plugged to tbf_profile_media_images and pass the proper file ids from the included files.
For the file and media migration jobs, I used the same source script called MediaGenericContent and here is how that looks:
<?php
namespace Drupal\tbf_migrate\Plugin\migrate\source\d7;
use Drupal\Core\Extension\ModuleHandlerInterface;
use Drupal\migrate\Row;
use Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity;
use Drupal\Core\Database\Query\SelectInterface;
use Drupal\Core\Entity\EntityManagerInterface;
use Drupal\Core\Extension\ModuleHandler;
use Drupal\Core\State\StateInterface;
use Drupal\migrate\Plugin\MigrationInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
/**
* Drupal 7 file source from database.
*
* @MigrateSource(
* id = “tbf_media_generic_content”,
* source_module = “node”
* )
*/
class MediaGenericContent extends FieldableEntity {
/**
* The module handler.
*
* @var \Drupal\Core\Extension\ModuleHandlerInterface
*/
protected $moduleHandler;
/**
* {@inheritdoc}
*/
public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, StateInterface $state, EntityManagerInterface $entity_manager, ModuleHandlerInterface $module_handler) {
parent::__construct($configuration, $plugin_id, $plugin_definition, $migration, $state, $entity_manager);
$this->moduleHandler = $module_handler;
}
/**
* {@inheritdoc}
*/
public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) {
return new static(
$configuration,
$plugin_id,
$plugin_definition,
$migration,
$container->get(‘state’),
$container->get(‘entity.manager’),
$container->get(‘module_handler’)
);
}
/**
* The join options between the node and the node_revisions table.
*/
const JOIN = ‘n.vid = nr.vid’;
/**
* {@inheritdoc}
*/
public function query() {
// Select node in its last revision.
$query = $this->select(‘node_revision’, ‘nr’)
->fields(‘n’, [
‘nid’,
])
->fields(‘nr’, [
‘title’,
]);
$query->addField(‘n’, ‘uid’, ‘node_uid’);
$query->addField(‘nr’, ‘uid’, ‘revision_uid’);
$query->innerJoin(‘node’, ‘n’, static::JOIN);
// If the content_translation module is enabled, get the source langcode
// to fill the content_translation_source field.
if ($this->moduleHandler->moduleExists(‘content_translation’)) {
$query->leftJoin(‘node’, ‘nt’, ‘n.tnid = nt.nid’);
$query->addField(‘nt’, ‘language’, ‘source_langcode’);
}
$this->handleTranslations($query);
// Add user email to match UID to the current Drupal created ones
$query->leftJoin(‘users’, ‘u’, ‘u.uid = n.uid’);
$query->addField(‘u’, ‘mail’, ‘author’);
if (isset($this->configuration[‘bundles’])) {
$query->condition(‘n.type’, $this->configuration[‘bundles’], ‘IN’);
}
// Add content type label
if (isset($this->configuration[‘field_name’]) && !empty($this->configuration[‘field_name’])) {
$file_field = $this->configuration[‘field_name’];
$query->leftJoin(‘field_revision_’.$file_field, ‘ff’, ‘ff.revision_id = n.vid’);
$query->leftJoin(‘file_managed’, ‘fm’, ‘fm.fid = ff.’.$file_field.’_fid’);
$query->addField(‘fm’, ‘fid’, ‘fid’);
$query->addField(‘fm’, ‘uri’, ‘file_path’);
$query->addField(‘fm’, ‘filename’, ‘file_name’);
if (isset($this->configuration[‘media_type’])) {
switch ($this->configuration[‘media_type’]) {
case ‘image’:
default:
$query->addField(‘ff’, $file_field.’_alt’, ‘file_alt’);
$query->addField(‘ff’, $file_field.’_title’, ‘file_title’);
}
}
$query->condition(‘fm.filename’, ”, ‘!=’);
} else {
return false;
}
return $query;
}
/**
* {@inheritdoc}
*/
public function prepareRow(Row $row) {
dpm($row);
$nid = $row->getSourceProperty(‘nid’);
$vid = $row->getSourceProperty(‘vid’);
$type = $row->getSourceProperty(‘type’);
$file_name = $row->getSourceProperty(‘file_name’);
$source = $row->getSource();
// If this entity was translated using Entity Translation, we need to get
// its source language to get the field values in the right language.
// The translations will be migrated by the d7_node_entity_translation
// migration.
$entity_translatable = $this->isEntityTranslatable(‘node’) && (int) $this->variableGet(‘language_content_type_’ . $type, 0) === 4;
$source_language = $this->getEntityTranslationSourceLanguage(‘node’, $nid);
$language = $entity_translatable && $source_language ? $source_language : $row->getSourceProperty(‘language’);
// Get Field API field values.
foreach ($this->getFields(‘node’, $type) as $field_name => $field) {
// Ensure we’re using the right language if the entity and the field are
// translatable.
$field_language = $entity_translatable && $field[‘translatable’] ? $language : NULL;
$row->setSourceProperty($field_name, $this->getFieldValues(‘node’, $field_name, $nid, $vid, $field_language));
}
// Make sure we always have a translation set.
if ($row->getSourceProperty(‘tnid’) == 0) {
$row->setSourceProperty(‘tnid’, $row->getSourceProperty(‘nid’));
}
// If the node title was replaced by a real field using the Drupal 7 Title
// module, use the field value instead of the node title.
if ($this->moduleExists(‘title’)) {
$title_field = $row->getSourceProperty(‘title_field’);
if (isset($title_field[0][‘value’])) {
$row->setSourceProperty(‘title’, $title_field[0][‘value’]);
}
}
// Point authoring info to the right User
$mail = $row->getSourceProperty(‘author’);
$uid = 1;
$users = \Drupal::entityTypeManager()->getStorage(‘user’)->loadByProperties([‘mail’ => $mail]);
$user = reset($users);
if ($user) {
$uid = $user->id();
}
$row->setSourceProperty(‘uid’, $uid);
return parent::prepareRow($row);
}
/**
* {@inheritdoc}
*/
public function fields() {
$fields = [
‘fid’ => $this->t(‘The file entity ID.’),
‘nid’ => $this->t(‘The file entity appearance NID.’), // Adding fid and nid to prevent media overwrite
// ‘file_id’ => $this->t(‘The file entity ID.’),
‘file_path’ => $this->t(‘The file path.’),
‘file_name’ => $this->t(‘The file name.’),
‘file_alt’ => $this->t(‘The file arl.’),
‘file_title’ => $this->t(‘The file title.’),
‘uid’ => $this->t(‘User ID.’),
];
return $fields;
}
/**
* {@inheritdoc}
*/
public function getIds() {
// Adding fid ID next to nid to prevent media overwrite.
$ids[‘fid’][‘type’] = ‘integer’;
$ids[‘fid’][‘alias’] = ‘f’;
$ids[‘nid’][‘type’] = ‘integer’;
$ids[‘nid’][‘alias’] = ‘n’;
return $ids;
}
}
Our last challenge was to have a given node type that contained multiple taxonomy term references plus multiple node references.
The migration API offers their own processor for enumerated values.
In order to process all references, we take them from the source class, then pass the comma-separated list of ids through a processor for them to be mapped as arrays within the migration process.
For the following migration we needed to catch multiple terms from a previously migrated vocabulary:
id: tbf_reports_content
label: Report/s Content
migration_group: tbf_content
audit: true
migration_tags:
– Drupal 7
– Content
source:
plugin: tbf_node_generic_content
bundles:
– reports
taxonomy_fields:
– field_tags
media_fields:
– field_image
– field_pdf
process:
langcode:
plugin: default_value
source: language
default_value: “und”
title: title
uid: node_uid
status: status
created: created
changed: changed
promote: promote
sticky: sticky
revision_uid: revision_uid
revision_log: log
revision_timestamp: timestamp
# Body
‘body/format’:
plugin: default_value
default_value: ‘full_html’
‘body/value’: body_value
‘body/summary’: body_summary
‘field_header/target_id’:
plugin: migration_lookup
migration: tbf_blog_header_reports
source: nid
no_stub: true
‘field_image/target_id’:
plugin: migration_lookup
migration: tbf_blog_media_images
source: field_image_fid
no_stub: true
‘field_pdf/target_id’:
plugin: migration_lookup
migration: tbf_blog_media_pdf
source: field_pdf_fid
no_stub: true
# field_state is called field_tags in this content type
state_list:
plugin: term_parser
source: field_tags_tid
field_state:
plugin: sub_process
source: ‘@state_list’
process:
target_id:
plugin: migration_lookup
migration: tbf_states_terms
source: tid
# Content type (adjusted to D8)
type:
plugin: default_value
default_value: reports
destination:
plugin: entity:node
bundle: reports
migration_dependencies:
required:
– d7_user
– tbf_states_terms
– tbf_blog_media_images
– tbf_blog_media_pdf
Focus on the state_list/field_state part. The idea here is to take a list of comma-separated tid’s from our NodeGenericContent source class. After that, we will use a custom process class called TermParser to explode this into an array:
<?php
namespace Drupal\tbf_migrate\Plugin\migrate\process;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\Row;
use Drupal\migrate\Plugin\MigrateProcessInterface;
/**
* @MigrateProcessPlugin(
* id = “term_parser”,
* )
*/
class TermParser extends ProcessPluginBase implements MigrateProcessInterface {
/**
* {@inheritdoc}
*/
public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
$field_id = $this->configuration[‘source’];
$tid_list = $row->getSourceProperty($field_id);
$tids = [];
if (isset($tid_list) && !empty($tid_list)) {
$tid_list_array = explode(‘,’, $tid_list);
foreach ($tid_list_array as $tid) {
$tids[] = [‘tid’ => $tid];
}
}
return $tids;
}
}
After splitting the tids’ comma-separated string into an array, the sub_process plugin (bundled in Drupal) performs a loop through the states_list array and creates field instances with the values.
Since migration definitions (the YAML files) are stored as settings on Drupal, every time we perform an update, the drush cim (partial) command needs to take these changes as follows.:
drush cim –partial –source=/var/www/web/modules/custom/tbf_migrate/config/install
Where the “install” folder contains all our custom migration definitions.
This article provides one approach on how to handle a Drupal 7 to 8 migration. This is not the only way. The Migration API is definitely a great solution for importing content from different sources and we did not cover its full potential!
A good tip when starting to work with different migrations is to take a look at the YAML files and PHP migration classes that appear in the core migration modules, there could be a class that fits your needs or you can use one as a sample to create your own.
Another aspect to consider is that migrations can also be grouped, this way we can execute a migration group instead of separate migration jobs.
I also suggest going to Drupal.org, there are many useful articles and a more extensive overview about migrations for you to check out.
Sign up today to have our latest posts delivered straight to your inbox.
