Regression: Migrations table already exists error in 3.4

[uncaught]

Bug Report

Q	A
Version	3.4.0
Previous Version if the bug is a regression	3.3.1

Summary

After upgrading from 3.3.1 to 3.4.0 our migrations fail with

[Doctrine\DBAL\Exception\TableExistsException (1050)]
An exception occurred while executing a query: SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'doctrine_migration_versions' already exists

Full output and trace

15:06:22 NOTICE    [app] Running Doctrine migrations ...
15:06:22 CRITICAL  [console] Error thrown while running command "-vv hc:install". Message: "An exception occurred while executing a query: SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'doctrine_migration_versions' already exists" ["exception" => Doctrine\DBAL\Exception\TableExistsException^ { …},"command" => "-vv hc:install","message" => "An exception occurred while executing a query: SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'doctrine_migration_versions' already exists"]

In ExceptionConverter.php line 38:

  [Doctrine\DBAL\Exception\TableExistsException (1050)]
  An exception occurred while executing a query: SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'doctrine_migration_versions' already exists


Exception trace:
  at /var/www/html/vendor/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php:38
 Doctrine\DBAL\Driver\API\MySQL\ExceptionConverter->convert() at /var/www/html/vendor/doctrine/dbal/src/Connection.php:1414
 Doctrine\DBAL\Connection->handleDriverException() at /var/www/html/vendor/doctrine/dbal/src/Connection.php:1356
 Doctrine\DBAL\Connection->convertExceptionDuringQuery() at /var/www/html/vendor/doctrine/dbal/src/Connection.php:888
 Doctrine\DBAL\Connection->executeStatement() at /var/www/html/packages/doctrine/src/DBAL/Con.php:55
 Octaved\Doctrine\DBAL\Con->executeStatement() at /var/www/html/vendor/doctrine/dbal/src/Schema/AbstractSchemaManager.php:779
 Doctrine\DBAL\Schema\AbstractSchemaManager->executeStatements() at /var/www/html/vendor/doctrine/dbal/src/Schema/AbstractSchemaManager.php:514
 Doctrine\DBAL\Schema\AbstractSchemaManager->createTable() at /var/www/html/vendor/doctrine/migrations/src/Metadata/Storage/TableMetadataStorage.php:172
 Doctrine\Migrations\Metadata\Storage\TableMetadataStorage->ensureInitialized() at /var/www/html/vendor/doctrine/migrations/src/Tools/Console/Command/MigrateCommand.php:158
 Doctrine\Migrations\Tools\Console\Command\MigrateCommand->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:279
 Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Command/LazyCommand.php:77
 Symfony\Component\Console\Command\LazyCommand->run() at /var/www/html/packages/core/src/HContent/CoreBundle/Command/InstallCommand.php:67
 HContent\CoreBundle\Command\InstallCommand->runDoctrineMigrations() at /var/www/html/packages/core/src/HContent/CoreBundle/Command/InstallCommand.php:43
 HContent\CoreBundle\Command\InstallCommand->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:279
 Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Application.php:1094
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/framework-bundle/Console/Application.php:123
 Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:342
 Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/symfony/framework-bundle/Console/Application.php:77
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/www/html/vendor/symfony/console/Application.php:193
 Symfony\Component\Console\Application->run() at /var/www/html/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49
 Symfony\Component\Runtime\Runner\Symfony\ConsoleApplicationRunner->run() at /var/www/html/vendor/autoload_runtime.php:29
 require_once() at /var/www/html/app/api/bin/console:11

I could not find any upgrade notes that would hint this could happen in this minor update. Nor could I see anything obvious in the code changes between 3.3.1 and 3.4.0.

Is this a bug or did we something wrong?

We have a few middlewares registered, but they don't appear to be relevant here. And our custom SqlGenerator is not at play here, either.

Current behavior

Migrations table is tried to be created.

Expected behavior

Migrations table already exist and should not be created.

How to reproduce

Our config

doctrine_migrations:
  # List of namespace/path pairs to search for migrations, at least one required
  migrations_paths:
    'Octaved\Doctrine\MigrationVersions': '%hc.path.packages%/doctrine/migrations'

  storage:
    table_storage:
      table_name: 'doctrine_migration_versions'
      version_column_name: 'version'
      version_column_length: 191
      executed_at_column_name: 'executed_at'
      execution_time_column_name: 'execution_time'

  # Possible values: "BY_YEAR", "BY_YEAR_AND_MONTH", false
  organize_migrations: 'BY_YEAR' # this config is not used because we have overwritten the Generator

  # Run all migrations in a transaction.
  all_or_nothing: false

  # Adds an extra check in the generated migrations to ensure that is executed on the same database type.
  check_database_platform: false

  # Whether or not to wrap migrations in a single transaction.
  transactional: false

[greg0ire]

Have you seen #584 ? It's not the same issue, but the cause is probably the same.

[uncaught]

Yes, I've seen that, but didn't look like my problem.

I'm just guessing here: Maybe it happens because we call the migrate command from another command and this change here sets some sort of listener on an onConsoleCommand-event: 3.3.1...3.4.0#diff-1e740c1cd2c18e620cb682897cb371e90567eb612329fbfbd3cfa1a581972c74R149

Maybe there lies the culprit? Would nested application calls not be supported this way?

    $this->logger->notice('Running Doctrine migrations ...');
    /** @var MigrateCommand $command */
    $command = $this->getApplication()->find('doctrine:migrations:migrate');
    $input = new ArrayInput(['--allow-no-migration' => true, '']);
    $input->setInteractive(false);
    $quietOutput = new ConsoleOutput(OutputInterface::VERBOSITY_QUIET);
    $command->run($input, $quietOutput);

[Chris53897]

I can confirm this. We run it the same way and get the error.

[greg0ire]

Ok it's yet another thing to fix in the listener, not sure how the event system behaves in the case of nested commands. Can you check how many times the listener is called?

[uncaught]

SchemaFilterListener::onConsoleCommand is called once before we start the doctrine application.

And then there are 133 calls to SchemaFilterListener::__invoke before the exception is thrown.

[greg0ire]

Ah, if it's called once, then I don't foresee an easy solution to this.

[greg0ire]

Can you let me know if #587 fixes the issue?

[uncaught]

Can you let me know if #587 fixes the issue?

Yes, it does!