DrupalCon is always a great opportunity to share experiences, knowledge and some funny stories about our daily work.

What caught my attention last year was the reaction from fellow drupalers when I said I was using the Migration API to move content from Drupal 7 to Drupal 8. That made me feel that people should get more comfortable with this API (it doesn’t bite!) to take advantage of their features/benefits, rather than using custom data import methods.

Of course, there’s always a considerable learning curve around coding in Drupal 8. We all know about the changes / improvements between D8 and previous versions (I swear that moving stuff from Drupal 8 to 9 will be waaaay easier than this), so figuring migration out completely actually took me a few months.

Migration API is based on the following principles:

  • There is a source plugin where you perform the data extraction from a given resource (can be a Drupal < 9 site, a CSV file, a RSS input or an XML)

  • The process plugins allow you to alter or adjust the source data to compile your final destination’s specs

  • The destination plugin connects the resulting data with your brand-new Drupal site


Image: https://www.drupal.org/docs/8/api/migrate-api/migrate-api-overview

Drupal offers someextremely useful plugins (source, process, destination) out-of-the-box that I suggest you take a look at:

Here are a few of the most common issues I experienced using the Migrate API in Drupal 8:

  • Moving nodes with referenced terms

  • Moving media elements (images/audio/videos)

  • Referenced nodes/terms

What’s a migration?

A migration is a procedure/job, defined by a YAML file. These migration files are usually located in modules implementing the Migrate API, in the following path (for our module called “tbf_migrate”):

tbf_migrate/src/config/install/migrate_plus.migration.agency_terms.yml

This file provides all the instructions for the migration to be executed:

# Agency terms from D7

id: agency_terms

label: Agency Terms

migration_group: d7_content_group

source:

 plugin: term_content

 vocabulary_machine_name: agencies

process:

 vid:

   plugin: default_value

   default_value: agencies

 name: name

 description: description

 weight: weight

 changed: timestamp

destination:

 plugin: entity:taxonomy_term

migration_dependencies:




First Challenge: Migration Plan

In order to understand migrations and the proper way to implement them, we must go from specific to general. This means, first moving roles and users, then taxonomy content, media (and files), then nodes that are referenced by other nodes, then more complex nodes, referencing terms, media and other nodes and entities, and finally URLs (I understand there are many ways to do this last thing). 

Requirements

First of all, is recommended to add some required dependencies to your custom module. Here is an example of how the [module].info.yml might look.

tbf_migrate.info.yml:

name: TBF Migrate Helper

description: Performs Migration from D7 to D8.

package: Other

dependencies:

– migrate

– migrate_plus

– migrate_tools

type: module

core: 8.x

Adding the migration database information

As mentioned before, Migrations start with a source. For our implementation, we required the Drupal 7 database from the old site, and we add it as a secondary database to our settings.php file:

// Adding migrate database to handle migrations

$databases[‘migrate’][‘default’] = array(

 ‘driver’ => ‘mysql’,

 ‘host’ => ‘old-site.com’,

 ‘username’ => ‘migrator_user’,

 ‘password’ => ‘xxxxxxxxxxxxxx’,

 ‘database’ => ‘drupal_7_db’,

 ‘prefix’ => ”,

);

Depending on where you are running your migration (local or dev environment), you will need to make sure that you can access the data source.

Additional Settings

After setting the database configuration, we need to create a Migration Group, in our case we placed all migration files within our module folder:

tbf_migrate/config/install

Our migration group YAML file is called migrate_plus.migration_group.d7_content_group.yml (the migrate_plus prefix on every migration or migration_group file is required).

#id of the config

id: d7_content_group

label: Content Migration Group

description: Drupal 7 migrations group.

source_type: Drupal 7

# In this section we set a database connection defined in settings.php

# So all migrations in this group will use this source.

shared_configuration:

 source:

   key: migrate

# Here we set a dependency on the module itself.

# This is necessary setting that deletes configs from a database

# on uninstallation of the module.

dependencies:

 enforced:

   module:

     – tbf_migrate

You can create more migration groups by adding more YAML files with the specified structure.

Moving Users and Roles

Along with already packaged Migration API plugins, I suggest you review all the modules you have installed on your Drupal 8/9 site, because many of them have Migration API support and migration files that can be useful.

And this is the case with the built-in User module. It offers some migrations out-of-the-box, and I used two of them:

  • d7_user_roles

  • d7_user

And this is how we start migrating user info from the Drupal 7 database within our Drupal 8 folder using Drush:

drush migrate:import d7_user_roles

drush migrate:import d7_user

After running these commands, drush will return the status for the performed migration (imported/ignored row amounts).

Note: It is going to be easier to migrate a Drupal 7 site to Drupal 8. So if you are on Drupal 6, I would suggest first upgrading to Drupal 7. Likewise, before moving to Drupal 9, you will first want to upgrade to Drupal 8.

Moving simple nodes

For this one, I created my own migration plugins based on the existing d7_node source migration plugin. Then under the fields area I separated fields based on type (single fields, reference fields, media fields).

Here’s a sample migration for a really simple node type with title and body.

id: d7_basic_page_content

label: Basic Page Content

migration_group: d7_content_group

audit: true

migration_tags:

 – Drupal 7

 – Content

source:

 plugin: tbf_node_generic_content

 node_type: ‘page’

process:

 langcode:

   plugin: default_value

   source: language

   default_value: “und”

 title: title

 uid: node_uid

 status: status

 created: created

 changed: changed

 promote: promote

 sticky: sticky

 revision_uid: revision_uid

 revision_log: log

 revision_timestamp: timestamp

 type:

   plugin: content_type_mapper

   default_value: type

 ‘body/format’:

   plugin: default_value

   default_value: ‘full_html’

 ‘body/value’: body_value

 ‘body/summary’: body_summary

destination:

 plugin: entity:node

 bundle: basic_page

migration_dependencies:

 required:

   – d7_user

Let’s break this big YAML down to analyze every aspect of this migration:

  • id, label, migration_group: the YAML file opens with these required properties

  • source: for the source we created our own source plugin, and we passed a parameter called node_type for the plugin to filter only Basic Page contents. Migration source plugins are PHP classes previously created and stored in /tbf_migrate/src/Plugin/migrate/source/. We’ll talk about this more later.

  • process: in this part of the migration, we prepare all the information before storing it in our new Drupal 8 site. Here you can perform mappings, parsing, replacements, providing fixed values. As mentioned previously, Drupal offers a nice plugin set for handling the most common processing situations. In this case, the processing is simple because we are mapping most of the values to their destination, except for the langcode and body/format parts. Here we used one of the most common processing plugins called default_value to explicitly assign a value. Since our website doesn’t contain multiple languages, we set the langcode to be ‘und’ by default. Same for body/format, where its default is going to be ‘full_html’. Please notice there is another plugin, for the type entry. Here’s our first custom plugin called content_type_mapper, and we will share the details later.

  • migration_dependencies: Since the D8 destination had their content types created we didn’t need to use the d7_node_type migration job from the Node module. In this case, we kept the d7_user dependency because we need users migrated before any content migration for authoring purposes.

Two custom plugins

NodeGenericContent Source Class (tbf_node_generic_content)

Sometimes when the included migration plugins don’t fill your requirements, creating custom plugins is necessary. As mentioned many times here, you can pick an existing plugin and customize the code within your module. That’s what happened with my first sample called tbf_node_generic_content. For this plugin I took the code from the Node module (web/core/modules/node/src/Plugin/migrate/source/d7/Node.php), and copied it into the custom module folder we created (web/modules/custom/tbf_migrate/src/Plugin/migrate/source/d7/NodeGenericContent.php).

Note: This is a large chunk of code that I included for your reference. You can go through it and figure out what might be useful for your use case.

<?php

namespace Drupal\tbf_migrate\Plugin\migrate\source\d7;

use Drupal\Core\Extension\ModuleHandlerInterface;

use Drupal\migrate\Row;

use Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity;

use Drupal\Core\Database\Query\SelectInterface;

use Drupal\Core\Entity\EntityManagerInterface;

use Drupal\Core\Extension\ModuleHandler;

use Drupal\Core\State\StateInterface;

use Drupal\migrate\Plugin\MigrationInterface;

use Symfony\Component\DependencyInjection\ContainerInterface;

/**

* Drupal 7 node source from database.

*

* @MigrateSource(

*   id = “tbf_node_generic_content”,

*   source_module = “node”

* )

*/

class NodeGenericContent extends FieldableEntity {

 /**

  * The module handler.

  *

  * @var \Drupal\Core\Extension\ModuleHandlerInterface

  */

 protected $moduleHandler;

 /**

  * {@inheritdoc}

  */

 public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, StateInterface $state, EntityManagerInterface $entity_manager, ModuleHandlerInterface $module_handler) {

   parent::__construct($configuration, $plugin_id, $plugin_definition, $migration, $state, $entity_manager);

   $this->moduleHandler = $module_handler;

 }

 /**

  * {@inheritdoc}

  */

 public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) {

   return new static(

     $configuration,

     $plugin_id,

     $plugin_definition,

     $migration,

     $container->get(‘state’),

     $container->get(‘entity.manager’),

     $container->get(‘module_handler’)

   );

 }

 /**

  * The join options between the node and the node_revisions table.

  */

 const JOIN = ‘n.vid = nr.vid’;

 /**

  * {@inheritdoc}

  */

 public function query() {

   // Select node in its last revision.

   $query = $this->select(‘node_revision’, ‘nr’)

     ->distinct(true)

     ->fields(‘n’, [

       ‘nid’,

       ‘type’,

       ‘language’,

       ‘status’,

       ‘created’,

       ‘changed’,

       ‘comment’,

       ‘promote’,

       ‘sticky’,

       ‘tnid’,

       ‘translate’,

     ])

     ->fields(‘nr’, [

       ‘vid’,

       ‘title’,

       ‘log’,

       ‘timestamp’,

     ]);

   $query->addField(‘n’, ‘uid’, ‘node_uid’);

   $query->addField(‘nr’, ‘uid’, ‘revision_uid’);

   $query->innerJoin(‘node’, ‘n’, static::JOIN);

   // If the content_translation module is enabled, get the source langcode

   // to fill the content_translation_source field.

   if ($this->moduleHandler->moduleExists(‘content_translation’)) {

     $query->leftJoin(‘node’, ‘nt’, ‘n.tnid = nt.nid’);

     $query->addField(‘nt’, ‘language’, ‘source_langcode’);

   }

   $this->handleTranslations($query);

   // Add user email to match UID to the current Drupal created ones

   $query->leftJoin(‘users’, ‘u’, ‘u.uid = n.uid’);

   $query->addField(‘u’, ‘mail’, ‘author’);

   if (isset($this->configuration[‘node_type’])) {

     $query->condition(‘n.type’, $this->configuration[‘node_type’]);

   }

   if (isset($this->configuration[‘bundles’]) && count($this->configuration[‘bundles’])) {

     $query->condition(‘n.type’, $this->configuration[‘bundles’], ‘IN’);

   }

   // Add content type label

   $query->leftJoin(‘node_type’, ‘nt’, ‘n.type = nt.type’);

   $query->addField(‘nt’, ‘name’, ‘type_label’);

   $query->leftJoin(‘field_revision_body’, ‘nb’, ‘nb.revision_id = n.vid’);

   $query->addField(‘nb’, ‘body_value’, ‘body_value’);

   $query->addField(‘nb’, ‘body_summary’, ‘body_summary’);

   // We need to set some key fields in the groupBy function

   $group_concat_used = false;

   $groupBy = [];

   $groupBy = [‘n.nid’, ‘nb.body_value’, ‘nb.body_summary’];

   // Adding fields from source config

   // Basic fields – Fields containing values in revision table.

   if (isset($this->configuration[‘basic_fields’])) {

     $i = 0;

     foreach ($this->configuration[‘basic_fields’] as $field) {

       $query->leftJoin(‘field_revision_’.$field, ‘f’.$i, ‘f’.$i.’.revision_id = n.vid’);

       $query->addField(‘f’.$i, $field.’_value’, $field);

       $groupBy[] = ‘f’.$i.’.’.$field.’_value’;

       $i++;

     }

   }

   if (isset($this->configuration[‘taxonomy_fields’])) {

     $i = 0;

     foreach ($this->configuration[‘taxonomy_fields’] as $field) {

       $query->leftJoin(‘field_revision_’.$field, ‘t’.$i, ‘t’.$i.’.revision_id = n.vid’);

       $query->addExpression(‘GROUP_CONCAT(DISTINCT(t’.$i.’.’.$field.’_tid))’, $field.’_tid’);

       $group_concat_used = true;

       $i++;

     }

   }

   if (isset($this->configuration[‘entity_fields’])) {

     $i = 0;

     foreach ($this->configuration[‘entity_fields’] as $field) {

       $query->leftJoin(‘field_revision_’.$field, ‘e’.$i, ‘e’.$i.’.revision_id = n.vid’);

       $query->addExpression(‘GROUP_CONCAT(DISTINCT(e’.$i.’.’.$field.’_target_id))’, $field.’_target_id’);

       $group_concat_used = true;

       $i++;

     }

   }

   if (isset($this->configuration[‘media_fields’])) {

     $i = 0;

     foreach ($this->configuration[‘media_fields’] as $field) {

       $query->leftJoin(‘field_revision_’.$field, ‘r’.$i, ‘r’.$i.’.revision_id = n.vid’);

       $query->addField(‘r’.$i, $field.’_fid’, $field.’_fid’);

       $groupBy[] = ‘r’.$i.’.’.$field.’_fid’;

       $i++;

     }

   }

   if ($group_concat_used || count($groupBy))

     $query->groupBy(implode(‘, ‘, $groupBy));

   return $query;

 }

 /**

  * {@inheritdoc}

  */

 public function prepareRow(Row $row) {

   $nid = $row->getSourceProperty(‘nid’);

   $vid = $row->getSourceProperty(‘vid’);

   $type = $row->getSourceProperty(‘type’);

   // If this entity was translated using Entity Translation, we need to get

   // its source language to get the field values in the right language.

   // The translations will be migrated by the d7_node_entity_translation

   // migration.

   $entity_translatable = $this->isEntityTranslatable(‘node’) && (int) $this->variableGet(‘language_content_type_’ . $type, 0) === 4;

   $source_language = $this->getEntityTranslationSourceLanguage(‘node’, $nid);

   $language = $entity_translatable && $source_language ? $source_language : $row->getSourceProperty(‘language’);

   // Get Field API field values.

   foreach ($this->getFields(‘node’, $type) as $field_name => $field) {

     // Ensure we’re using the right language if the entity and the field are

     // translatable.

     $field_language = $entity_translatable && $field[‘translatable’] ? $language : NULL;

     $row->setSourceProperty($field_name, $this->getFieldValues(‘node’, $field_name, $nid, $vid, $field_language));

   }

   // Make sure we always have a translation set.

   if ($row->getSourceProperty(‘tnid’) == 0) {

     $row->setSourceProperty(‘tnid’, $row->getSourceProperty(‘nid’));

   }

   // If the node title was replaced by a real field using the Drupal 7 Title

   // module, use the field value instead of the node title.

   if ($this->moduleExists(‘title’)) {

     $title_field = $row->getSourceProperty(‘title_field’);

     if (isset($title_field[0][‘value’])) {

       $row->setSourceProperty(‘title’, $title_field[0][‘value’]);

     }

   }

   // Point authoring info to the right User

   $mail = $row->getSourceProperty(‘author’);

   $uid = 1;

   $users = \Drupal::entityTypeManager()->getStorage(‘user’)->loadByProperties([‘mail’ => $mail]);

   $user = reset($users);

   if ($user) {

     $uid = $user->id();

   }

   $row->setSourceProperty(‘uid’, $uid);

   return parent::prepareRow($row);

 }

 /**

  * {@inheritdoc}

  */

 public function fields() {

   $fields = [

     ‘nid’ => $this->t(‘Node ID’),

     ‘type’ => $this->t(‘Type’),

     ‘type_label’ => $this->t(‘Type Label’),

     ‘title’ => $this->t(‘Title’),

     ‘body_value’ => $this->t(‘Body’),

     ‘body_summary’ => $this->t(‘Summary’),

     ‘node_uid’ => $this->t(‘Node authored by (uid)’),

     ‘revision_uid’ => $this->t(‘Revision authored by (uid)’),

     ‘created’ => $this->t(‘Created timestamp’),

     ‘changed’ => $this->t(‘Modified timestamp’),

     ‘status’ => $this->t(‘Published’),

     ‘promote’ => $this->t(‘Promoted to front page’),

     ‘sticky’ => $this->t(‘Sticky at top of lists’),

     ‘revision’ => $this->t(‘Create new revision’),

     ‘language’ => $this->t(‘Language (fr, en, …)’),

     ‘tnid’ => $this->t(‘The translation set id for this node’),

     ‘timestamp’ => $this->t(‘The timestamp the latest revision of this node was created.’),

   ];

   // Basic fields – Fields containing values in revision table.

   if (isset($this->configuration[‘basic_fields’])) {

     foreach ($this->configuration[‘basic_fields’] as $field) {

       $fields[$field] = $field;

     }

   }

   if (isset($this->configuration[‘taxonomy_fields’])) {

     foreach ($this->configuration[‘taxonomy_fields’] as $field) {

       $fields[$field] = $field;

     }

   }

   if (isset($this->configuration[‘entity_fields’])) {

     foreach ($this->configuration[‘entity_fields’] as $field) {

       $fields[$field] = $field;

     }

   }

   if (isset($this->configuration[‘media_fields’])) {

     foreach ($this->configuration[‘media_fields’] as $field) {

       $fields[$field] = $field;

       $fields[$field.’_fid’] = $field.’_fid’;

     }

   }

   return $fields;

 }

 /**

  * {@inheritdoc}

  */

 public function getIds() {

   $ids[‘nid’][‘type’] = ‘integer’;

   $ids[‘nid’][‘alias’] = ‘n’;

   return $ids;

 }

 /**

  * Adapt our query for translations.

  *

  * @param \Drupal\Core\Database\Query\SelectInterface $query

  *   The generated query.

  */

 protected function handleTranslations(SelectInterface $query) {

   // Check whether or not we want translations.

   if (empty($this->configuration[‘translations’])) {

     // No translations: Yield untranslated nodes, or default translations.

     $query->where(‘n.tnid = 0 OR n.tnid = n.nid’);

   }

   else {

     // Translations: Yield only non-default translations.

     $query->where(‘n.tnid <> 0 AND n.tnid <> n.nid’);

   }

 }

}

The addition to this plugin class was the field management code, this way we only use one migration to bring all the contents, term, media and node reference fields, and other field types.

ContentTypeMapper Process Class (content_type_mapper)

The new D8 site needed to handle Content Types so that blog posts, press releases, letters and presentations would go into one general content type called “news”. For this reason, we created a process class to map content types to target the new machine names. 

Process plugins go under the src/Plugin/migrate/process folder. Our sample is located in

web/modules/custom/tbf_migrate/src/Plugin/migrate/process/ContentTypeMapper.php

<?php

namespace Drupal\tbf_migrate\Plugin\migrate\process;

use Drupal\migrate\MigrateExecutableInterface;

use Drupal\migrate\ProcessPluginBase;

use Drupal\migrate\Row;

use Drupal\migrate\Plugin\MigrateProcessInterface;

use \Drupal\Component\Utility\Html;

/**

* @MigrateProcessPlugin(

*   id = “content_type_mapper”,

* )

*/

class ContentTypeMapper extends ProcessPluginBase implements MigrateProcessInterface {

 /**

  * {@inheritdoc}

  */

 public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {

   $type = $row->getSourceProperty(‘type’);

   switch ($type) {

     case ‘page’:

       $type = ‘basic_page’;

       break;

     case ‘blog’:

     case ‘latest_news’:

     case ‘presentation’:

     case ‘press_release’:

       $type = ‘blog’;

       break;

     case ‘litigation’:

       $type = ‘tracker_litigation’;

       break;

     case ‘order_action’:

       $type = ‘tracker_order_action’;

       break;

     default:

       break;

   }

   return $type;

 }

}




Second Challenge: Complex nodes

Now that we have migrated some simple entities, let’s go to the next level. There were some nodes in Drupal 8 that had media entities. In this case, we need to create some additional migrations to handle the conversion of Drupal 7 file/image fields to media entities in Drupal 8.

Nodes with Media

We had a “Profile” content type that contained a related image so we had to:

  1. Move the files from the old D7 to the new D8’s files folder

  2. Create a File migration job 

  3. Create a Media migration job

  4. Then link the Media migration job to the Profile node migration job

The File migration YAML file is the following:

id: tbf_profile_images_files

label: Profile Image Files

migration_group: tbf_content

source:

 plugin: tbf_media_generic_content

 entity_type: node

 bundle: profile

 langcode: und

 field_name: field_profile_image

destination:

 plugin: entity:file

process:

 uid:

   plugin: default_value

   default_value: 1

 filename: file_name

 file_path: file_path

 uri:

   plugin: file_copy

   source:

     – ‘file_path’

     – ‘file_path’

   file_exists: use existing

migration_dependencies:

And the Media migration YAML file looks similar:

id: tbf_profile_media_images

label: Profile Image Media

migration_group: tbf_content

source:

 plugin: tbf_media_generic_content

 entity_type: node

 bundle: profile

 langcode: und

 field_name: field_profile_image

destination:

 plugin: entity:media

process:

 bundle:

  plugin: default_value

  default_value: image

 name: title

  uid:

   plugin: default_value

   default_value: 1

 ‘field_media_image/target_id’:

   plugin: migration_lookup

   migration: tbf_profile_images_files

   source: fid

 ‘field_media_image/alt’: file_name

 ‘field_media_image/title’: file_name

migration_dependencies:

 required:

 – tbf_profile_images_files

The main difference between the previous migrations is that tbf_profile_images_files needs to be processed first, then must be plugged to tbf_profile_media_images and pass the proper file ids from the included files.

More source and process plugins

For the file and media migration jobs, I used the same source script called MediaGenericContent and here is how that looks:

<?php

namespace Drupal\tbf_migrate\Plugin\migrate\source\d7;

use Drupal\Core\Extension\ModuleHandlerInterface;

use Drupal\migrate\Row;

use Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity;

use Drupal\Core\Database\Query\SelectInterface;

use Drupal\Core\Entity\EntityManagerInterface;

use Drupal\Core\Extension\ModuleHandler;

use Drupal\Core\State\StateInterface;

use Drupal\migrate\Plugin\MigrationInterface;

use Symfony\Component\DependencyInjection\ContainerInterface;

/**

* Drupal 7 file source from database.

*

* @MigrateSource(

*   id = “tbf_media_generic_content”,

*   source_module = “node”

* )

*/

class MediaGenericContent extends FieldableEntity {

 /**

  * The module handler.

  *

  * @var \Drupal\Core\Extension\ModuleHandlerInterface

  */

 protected $moduleHandler;

 /**

  * {@inheritdoc}

  */

 public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, StateInterface $state, EntityManagerInterface $entity_manager, ModuleHandlerInterface $module_handler) {

   parent::__construct($configuration, $plugin_id, $plugin_definition, $migration, $state, $entity_manager);

   $this->moduleHandler = $module_handler;

 }

 /**

  * {@inheritdoc}

  */

 public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) {

   return new static(

     $configuration,

     $plugin_id,

     $plugin_definition,

     $migration,

     $container->get(‘state’),

     $container->get(‘entity.manager’),

     $container->get(‘module_handler’)

   );

 }

 /**

  * The join options between the node and the node_revisions table.

  */

 const JOIN = ‘n.vid = nr.vid’;

 /**

  * {@inheritdoc}

  */

 public function query() {

   // Select node in its last revision.

   $query = $this->select(‘node_revision’, ‘nr’)

     ->fields(‘n’, [

       ‘nid’,

     ])

     ->fields(‘nr’, [

       ‘title’,

     ]);

   $query->addField(‘n’, ‘uid’, ‘node_uid’);

   $query->addField(‘nr’, ‘uid’, ‘revision_uid’);

   $query->innerJoin(‘node’, ‘n’, static::JOIN);

   // If the content_translation module is enabled, get the source langcode

   // to fill the content_translation_source field.

   if ($this->moduleHandler->moduleExists(‘content_translation’)) {

     $query->leftJoin(‘node’, ‘nt’, ‘n.tnid = nt.nid’);

     $query->addField(‘nt’, ‘language’, ‘source_langcode’);

   }

   $this->handleTranslations($query);

   // Add user email to match UID to the current Drupal created ones

   $query->leftJoin(‘users’, ‘u’, ‘u.uid = n.uid’);

   $query->addField(‘u’, ‘mail’, ‘author’);

   if (isset($this->configuration[‘bundles’])) {

     $query->condition(‘n.type’, $this->configuration[‘bundles’], ‘IN’);

   }

   // Add content type label

   if (isset($this->configuration[‘field_name’]) && !empty($this->configuration[‘field_name’])) {

     $file_field = $this->configuration[‘field_name’];

     $query->leftJoin(‘field_revision_’.$file_field, ‘ff’, ‘ff.revision_id = n.vid’);

     $query->leftJoin(‘file_managed’, ‘fm’, ‘fm.fid = ff.’.$file_field.’_fid’);

     $query->addField(‘fm’, ‘fid’, ‘fid’);

     $query->addField(‘fm’, ‘uri’, ‘file_path’);

     $query->addField(‘fm’, ‘filename’, ‘file_name’);

     if (isset($this->configuration[‘media_type’])) {

       switch ($this->configuration[‘media_type’]) {

         case ‘image’:

         default:

           $query->addField(‘ff’, $file_field.’_alt’, ‘file_alt’);

           $query->addField(‘ff’, $file_field.’_title’, ‘file_title’);

       }

     }

     $query->condition(‘fm.filename’, ”, ‘!=’);

   } else {

     return false;

   }

   return $query;

 }

 /**

  * {@inheritdoc}

  */

 public function prepareRow(Row $row) {

   dpm($row);

   $nid = $row->getSourceProperty(‘nid’);

   $vid = $row->getSourceProperty(‘vid’);

   $type = $row->getSourceProperty(‘type’);

   $file_name = $row->getSourceProperty(‘file_name’);

   $source = $row->getSource();

   // If this entity was translated using Entity Translation, we need to get

   // its source language to get the field values in the right language.

   // The translations will be migrated by the d7_node_entity_translation

   // migration.

   $entity_translatable = $this->isEntityTranslatable(‘node’) && (int) $this->variableGet(‘language_content_type_’ . $type, 0) === 4;

   $source_language = $this->getEntityTranslationSourceLanguage(‘node’, $nid);

   $language = $entity_translatable && $source_language ? $source_language : $row->getSourceProperty(‘language’);

   // Get Field API field values.

   foreach ($this->getFields(‘node’, $type) as $field_name => $field) {

     // Ensure we’re using the right language if the entity and the field are

     // translatable.

     $field_language = $entity_translatable && $field[‘translatable’] ? $language : NULL;

     $row->setSourceProperty($field_name, $this->getFieldValues(‘node’, $field_name, $nid, $vid, $field_language));

   }

   // Make sure we always have a translation set.

   if ($row->getSourceProperty(‘tnid’) == 0) {

     $row->setSourceProperty(‘tnid’, $row->getSourceProperty(‘nid’));

   }

   // If the node title was replaced by a real field using the Drupal 7 Title

   // module, use the field value instead of the node title.

   if ($this->moduleExists(‘title’)) {

     $title_field = $row->getSourceProperty(‘title_field’);

     if (isset($title_field[0][‘value’])) {

       $row->setSourceProperty(‘title’, $title_field[0][‘value’]);

     }

   }

   // Point authoring info to the right User

   $mail = $row->getSourceProperty(‘author’);

   $uid = 1;

   $users = \Drupal::entityTypeManager()->getStorage(‘user’)->loadByProperties([‘mail’ => $mail]);

   $user = reset($users);

   if ($user) {

     $uid = $user->id();

   }

   $row->setSourceProperty(‘uid’, $uid);

   return parent::prepareRow($row);

 }

 /**

  * {@inheritdoc}

  */

 public function fields() {

   $fields = [

     ‘fid’ => $this->t(‘The file entity ID.’),

     ‘nid’ => $this->t(‘The file entity appearance NID.’), // Adding fid and nid to prevent media overwrite

     // ‘file_id’ => $this->t(‘The file entity ID.’),

     ‘file_path’ => $this->t(‘The file path.’),

     ‘file_name’ => $this->t(‘The file name.’),

     ‘file_alt’ => $this->t(‘The file arl.’),

     ‘file_title’ => $this->t(‘The file title.’),

     ‘uid’ => $this->t(‘User ID.’),

   ];

   return $fields;

 }

 /**

  * {@inheritdoc}

  */

 public function getIds() {

   // Adding fid ID next to nid to prevent media overwrite.

   $ids[‘fid’][‘type’] = ‘integer’;

   $ids[‘fid’][‘alias’] = ‘f’;

   $ids[‘nid’][‘type’] = ‘integer’;

   $ids[‘nid’][‘alias’] = ‘n’;

   return $ids;

 }

}




Third Challenge: Migrating multi-value fields

Our last challenge was to have a given node type that contained multiple taxonomy term references plus multiple node references.

The migration API offers their own processor for enumerated values.

In order to process all references, we take them from the source class, then pass the comma-separated list of ids through a processor for them to be mapped as arrays within the migration process.

For the following migration we needed to catch multiple terms from a previously migrated vocabulary:

id: tbf_reports_content

label: Report/s Content

migration_group: tbf_content

audit: true

migration_tags:

 – Drupal 7

 – Content

source:

 plugin: tbf_node_generic_content

 bundles:

   – reports

 taxonomy_fields:

   – field_tags

 media_fields:

   – field_image

   – field_pdf

process:

 langcode:

   plugin: default_value

   source: language

   default_value: “und”

 title: title

 uid: node_uid

 status: status

 created: created

 changed: changed

 promote: promote

 sticky: sticky

 revision_uid: revision_uid

 revision_log: log

 revision_timestamp: timestamp

 # Body

 ‘body/format’:

   plugin: default_value

   default_value: ‘full_html’

 ‘body/value’: body_value

 ‘body/summary’: body_summary

  ‘field_header/target_id’:

   plugin: migration_lookup

   migration: tbf_blog_header_reports

   source: nid

   no_stub: true

 ‘field_image/target_id’:

   plugin: migration_lookup

   migration: tbf_blog_media_images

   source: field_image_fid

   no_stub: true

 ‘field_pdf/target_id’:

   plugin: migration_lookup

   migration: tbf_blog_media_pdf

   source: field_pdf_fid

   no_stub: true

 # field_state is called field_tags in this content type

 state_list:

   plugin: term_parser

   source: field_tags_tid

 field_state:

   plugin: sub_process

   source: ‘@state_list’

   process:

     target_id:

       plugin: migration_lookup

       migration: tbf_states_terms

       source: tid

 # Content type (adjusted to D8)

 type:

   plugin: default_value

   default_value: reports

destination:

 plugin: entity:node

 bundle: reports

migration_dependencies:

 required:

   – d7_user

   – tbf_states_terms

   – tbf_blog_media_images

   – tbf_blog_media_pdf


 

Focus on the state_list/field_state part. The idea here is to take a list of comma-separated tid’s from our NodeGenericContent source class. After that, we will use a custom process class called TermParser to explode this into an array:

<?php

namespace Drupal\tbf_migrate\Plugin\migrate\process;

use Drupal\migrate\MigrateExecutableInterface;

use Drupal\migrate\ProcessPluginBase;

use Drupal\migrate\Row;

use Drupal\migrate\Plugin\MigrateProcessInterface;

/**

* @MigrateProcessPlugin(

*   id = “term_parser”,

* )

*/

class TermParser extends ProcessPluginBase implements MigrateProcessInterface {

 /**

  * {@inheritdoc}

  */

 public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {

   $field_id = $this->configuration[‘source’];

   $tid_list = $row->getSourceProperty($field_id);

   $tids = [];

   if (isset($tid_list) && !empty($tid_list)) {

     $tid_list_array = explode(‘,’, $tid_list);

     foreach ($tid_list_array as $tid) {

       $tids[] = [‘tid’ => $tid];

     }

   }

   return $tids;

 }

}

After splitting the tids’ comma-separated string into an array, the sub_process plugin (bundled in Drupal) performs a loop through the states_list array and creates field instances with the values.

Handling migration definition updates

Since migration definitions (the YAML files) are stored as settings on Drupal, every time we perform an update, the drush cim (partial) command needs to take these changes as follows.:

drush cim –partial –source=/var/www/web/modules/custom/tbf_migrate/config/install

Where the “install” folder contains all our custom migration definitions.




Final words

This article provides one approach on how to handle a Drupal 7 to 8 migration. This is not the only way. The Migration API is definitely a great solution for importing content from different sources and we did not cover its full potential!

A good tip when starting to work with different migrations is to take a look at the YAML files and PHP migration classes that appear in the core migration modules, there could be a class that fits your needs or you can use one as a sample to create your own.

Another aspect to consider is that migrations can also be grouped, this way we can execute a migration group instead of separate migration jobs.

I also suggest going to Drupal.org, there are many useful articles and a more extensive overview about migrations for you to check out.