CSV Export Missing Columns - Bug Report for AtoM 2.8.1

[matthewb...@gmail.com]

Just dropping this here for the developers....  

CSV Export Missing Columns - Bug Report for AtoM 2.8.1

  Summary

  CSV exports in AtoM 2.8.1 may be missing the majority of columns and data under certain conditions. Exports only contain 20 columns (mostly empty) instead of the expected 97 columns defined in QubitInformationObject.yml.

  Environment

  - AtoM Version: 2.8.1 (schema version 193)
  - OS: Ubuntu 24.04 LTS
  - PHP: 7.4
  - MySQL: 8.0

  Symptoms

  When running php symfony csv:export, the resulting CSV file shows:
  - Only 20 columns: legacyId, parentId, qubitParentSlug, accessionNumber, language, script, digitalObjectURI, digitalObjectChecksum, scriptOfDescription, publicationStatus, physicalObjectName, physicalObjectLocation, physicalObjectType,
  eventStartDates, eventEndDates, eventDescriptions, eventActorHistories, eventPlaces, slug, culture
  - All descriptive fields are empty (title, scopeAndContent, extentAndMedium, dates, etc.)
  - Database verification confirms the data exists in information_object_i18n table with proper titles and content

  Context / How I Encountered This

  I upgraded an AtoM 2.6 database to 2.8.1 through a problematic migration process where:
  - Initial upgrade from 2.6 (schema version 175) to 2.8 failed due to table conflicts (https://groups.google.com/g/ica-atom-users/c/zVQLw1G0XV0/m/N7pMi4ONBQAJ) 
  - Ran migrations individually (176-193) with some failures
  - Manually updated schema version in settings table to 193
  - Imported the resulting database into a fresh AtoM 2.8.1 installation

  After this process, CSV exports were broken as described above.

  Root Cause

  The bug is in /lib/task/export/csvExportInformationObjectsTask.class.php. The task instantiates csvInformationObjectExport and calls $writer->setOptions($options) but never calls $writer->setParams().

  This causes $this->params to remain uninitialized. When QubitFlatfileExport::exportResource() executes line 248:

  if (!$this->params['nonVisibleElementsIncluded']) {
      $this->getHiddenVisibleElementCsvHeaders();
  }

  The undefined array access causes PHP to evaluate this condition incorrectly, triggering the visibility filtering logic in getHiddenVisibleElementCsvHeaders() which removes most columns from the export.

  Code Analysis

  File: /lib/task/export/csvExportInformationObjectsTask.class.php (lines 64-85)

  Current code:
  $writer = new csvInformationObjectExport(
      $arguments['path'],
      $options['standard'],
      $options['rows-per-file']
  );

  $writer->user = $context->getUser();
  $writer->setOptions($options);
  // MISSING: $writer->setParams() call

  foreach ($rows as $row) {
      $writer->user->setCulture($row['culture']);
      $resource = QubitInformationObject::getById($row['id']);
      $writer->exportResource($resource);
      // ...
  }

  File: /lib/flatfile/QubitFlatfileExport.class.php

  The setParams() method exists (line 341) but is never invoked by any export task:
  public function setParams($params)
  {
      $this->params = $params;
  }

  When exportResource() is called without params being set, line 248 accesses an undefined array. Depending on PHP configuration and system state, this may cause the visibility filter logic to execute incorrectly, removing columns from
  $this->columnNames (lines 280-282).

  Fix

  Add the missing setParams() call in csvExportInformationObjectsTask.class.php after line 69:

  $writer->user = $context->getUser();
  $writer->setOptions($options);
  $writer->setParams([
      'nonVisibleElementsIncluded' => true,
      'objectType' => 'informationObject'
  ]);

  Reproduction Uncertainty

  I encountered this after a problematic database migration, so I'm uncertain if:
  1. This affects all AtoM 2.8.1 installations (latent bug)
  2. This only affects instances with corrupted/incomplete settings data
  3. This depends on specific PHP configuration (error handling for undefined array access)

  The code bug (missing setParams() call) definitely exists in AtoM 2.8.1, but it may not trigger symptoms in all installations. I'm reporting this so the developers can investigate whether this is a widespread issue or specific to certain
  conditions.

  Steps to Verify Fix

  1. Apply the fix above
  2. Run: php symfony cc
  3. Run: php symfony csv:export /path/to/export-fixed.csv
  4. Verify CSV now contains 97 columns with populated data:
  head -1 export-fixed.csv | tr ',' '\n' | wc -l
  # Should return 97

  Additional Notes

  - The setParams() method was likely introduced for the visibility filtering feature but the integration was never completed in the export tasks
  - A grep search confirms setParams() is defined but never called anywhere in the codebase:
  grep -rn "setParams" /usr/share/nginx/atom/lib/task/export/ --include="*.php"
  # Returns no results

 Let me know if you need additional details about my environment or upgrade process.