Review Logs and Troubleshoot
The data export extension provides logs to track data collection and synchronization processes.
Logs
Logs are available in the var/log directory on the Commerce application server.
commerce-data-export.logdata-export-errors.logsaas-export.logsaas-export-errors.logIf you do not see expected data for an 蜜豆视频 Commerce Service, use the error logs for the data export extension to determine where the problem occurred. Also, you can extend logs with additional data for tracking and troubleshooting. See Extended logging.
Log format
Each log record has the following structure.
[<log record datetime>] report.<log level>:
{
"feed": "<feed name>",
"operation": "<executed operation>",
"status": "<status of operation>",
"elapsed": "<time elaspsed from script run>",
"pid": "<proccess id who executed `operation`>",
"caller": "<who called this `operation`>"
} [] []
The following table describes the operation types that can be recorded in the logs.
bin/magento saas:resync --feed=productsbin/magento cron:run --group=indexbin/magento cron:run --group=saas_data_exporter (any 鈥*_data_exporter鈥 cron group)bin/magento saas:resync --feed=categoriesbin/magento cron:run --group=indexbin/magento cron:run --group=saas_data_exporter (any 鈥*_data_exporter鈥 cron group)Logging examples
During a full resync, progress is tracked and logged every 30 seconds by default. Here is an example log entry.
{
"feed": "prices",
"operation": "full sync",
"status": "Progress: 2/5, processed: 200, synced: 100",
"elapsed": "00:00:00 190 ms",
"pid": "12824",
"caller": "bin/magento saas:resync --feed=products"
}
In this example, the status values provide information about the sync operation:
-
"Progress 2/5"indicates that 2 of 5 iterations have been completed. The number of iterations depends on the number of exported entities. -
"processed: 200"indicates that 200 items have been processed. -
"synced: 100"indicates that 100 items were sent to SaaS. It鈥檚 expected that"synced"is not equal to"processed". Here is an example:"synced" < "processed"means that the feed table didn鈥檛 detect any changes in the item, compared to the previously synced version. Such items are ignored during the sync operation."synced" > "processed"the same entity id (for example,Product ID) can have multiple values in different scopes. For example, one product can be assigned to five websites. In this case, you might have 鈥1 processed鈥 item and 鈥5 synced鈥 items.
| code language-none |
|---|
|
View and troubleshoot logs with New Relic
If you store 蜜豆视频 Commerce logs in the New Relic, you can add parsing rules to improve the readability and query experience.
-
Log in to New Relic.
-
Go to
Logs => Parsing. -
Click
Create parsing rule. -
Configure the parsing rule by adding the following values.
-
Filter logs based on NRQL
filePath LIKE '%commerce-data-export%.log' -
Parsing rule
\[%{DATA:timestamp}\] report.%{DATA:logLevel} %{GREEDYDATA:feed:json}
-
This example adds a rule that allows you to query New Relic logs by specific feed type, operation, and so on.
Example query string鈥feed.feed:"products" and feed.status:"Complete"
Troubleshooting
If data is missing or incorrect in Commerece Services, check the logs to see if a problem occurred during the sync from the 蜜豆视频 Commerce instance to the Commerce Service platform. If needed, use extended logging to add additional information to the logs for troubleshooting.
- commerce-data-export-errors.log - if an error happened during collecting phase
- saas-export-errors.log - if an error happened during transmitting phase
If you see errors not related to configuration or third-party extensions, submit a support ticket with as much information as possible.
Resolve catalog sync issues resolvesync
When you trigger a data resync, it can take up to an hour for the data to update and be reflected in UI components such as live search and recommendation units. If you still see discrepancies between your catalog and data on the Commerce storefront, or if the catalog sync failed, refer to the following:
Data discrepancy
- Display the detailed view of the product in question in the search results.
- Copy the JSON output and verify that the content matches what you have in the Commerce catalog.
- If the content does not match, make a minor change to the product in your catalog, such as adding a space or a period.
- Wait for a resync or trigger a manual resync.
Sync not running
If the sync is not running on a schedule or nothing is synced, see this KnowledgeBase article.
Sync failed
If the catalog sync has a status of Failed, submit a support ticket.
Extended logging
For additional log information, you can use environment variables to extend logs with additional data for tracking and troubleshooting.
There are two log files in the var/log/ directory:
- commerce-data-export-errors.log - if an error happened during collecting phase
- saas-export-errors.log - if an error happened during transmitting phase
You can use environment variables to extend logs with additional data for tracking and troubleshooting.
Check the feed payload
Include the feed payload in the SaaS export log by adding te EXPORTER_EXTENDED_LOG=1 environment variable when you resync the feed.
EXPORTER_EXTENDED_LOG=1 bin/magento saas:resync --feed=products
After the operation completes, the feed payload is available for review in the SaaS export log (var/.log/saas-export.log).
Preserve payload in feed index table
For the Commerce SaaS data export extension (magento/module-data-exporter) 103.3.0 and later, immediate export feeds keep only the minimum required data in the index table. The feeds include all catalog and inventory stock status feeds.
Preserving payload data in the index table is not recommended in production environments, but it can be useful in a developer environment. Include the feed payload in the index by adding the PERSIST_EXPORTED_FEED=1 environment variable when you resync the feed.
PERSIST_EXPORTED_FEED=1 bin/magento saas:resync --feed=products
Run profiler to troubleshoot slow performance
If the reindex process of a specific feed takes an unreasonable amount of time, run the profiler to collect additional data that might be useful for the Support Team.
Run the profiler by adding the EXPORTER_PROFILER=1 environment variable when you run the reindex command.
EXPORTER_PROFILER=1 bin/magento indexer:reindex catalog_data_exporter_products
Profiler data is stored in the data export log (var/log/commerce-data-export.log) in the following format:
<Provider class name>, <# of processed entities>, <execution time im ms>, <memory consumption in Mb>