ÃÛ¶¹ÊÓƵ

DocumentationExperience PlatformAdvanced Data Lifecycle Management Guide

Automated dataset expirations dataset-expiration

Last update: Wed Jul 24 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

  • User
  • Developer
  • Admin

The Data Lifecycle workspace in the ÃÛ¶¹ÊÓƵ Experience Platform UI allows you to schedule expirations for datasets. When a dataset reaches its expiration date, the data lake, Identity Service, and Real-Time Customer Profile begin separate processes to remove the dataset’s contents from their respective services. Once the data is deleted from all three services, the expiration is marked as complete.

WARNING
If a dataset is set to expire, you must manually change any dataflows that may be ingesting data into that dataset so that your downstream workflows are not negatively affected.

This document covers how to schedule and automate dataset expirations in the Platform UI.

NOTE
Dataset Expiration does not currently delete data from the ÃÛ¶¹ÊÓƵ Experience Platform Edge Network. However, there is no possibility that data stays inside the Edge Network after the dataset is set to expire. This is because the 15-day service license agreement for Dataset Expiration overlaps with the 14-day period where data exists inside the Edge Network before being discarded.

Advanced Data Lifecycle Management supports dataset deletions through the dataset expiration endpoint and ID deletions (row-level data) using primary identities via the workorder endpoint. You can also manage dataset expirations and record deletions through the Platform UI. See the linked documentation for more information.

NOTE
Data Lifecycle does not support batch deletion.

Schedule a dataset expiration schedule-dataset-expiration

To create a request, select Create request from the main page in the workspace.

IMPORTANT
Real-Time CDP, ÃÛ¶¹ÊÓƵ Journey Optimizer, and Customer Journey Analytics users have 20 pending scheduled dataset expiration work orders. Healthcare Shield and Privacy and Security Shield users have 50 pending scheduled dataset expiration work orders. This means that you can have 20 or 50 datasets scheduled to be deleted at any one time.
For example, if you have 20 scheduled dataset expirations and one dataset is due to be deleted tomorrow, you cannot set any more expirations until after that that dataset has been deleted.

The Data Lifecycle workspace with Create request highlighted.

The request creation workflow appears. Under the Requested Action section, select Delete Dataset to update the controls for dataset expiration scheduling.

The request creation workflow with the Delete dataset option highlighted.

Select a date and a dataset select-date-and-dataset

Under the Requested Action section, select a date that you want the dataset to be deleted by. You can enter the date manually (in the format mm/dd/yyyy) or select the calendar icon ( A calendar icon. ) to select the date from a dialog.

A calendar dialog with an expiration date selected and highlighted for the dataset.

Next, under Dataset Details, select the database icon ( The database icon. ) to open a dataset selection dialog. Choose a dataset from the list to apply the expiration to, then select Done.

The Select dataset dialog with a dataset selected and Done highlighted.

NOTE
Only datasets belonging to the current sandbox are shown.

Submit the request submit-request

The Dataset Details section populates to include the primary identity and schema for the selected dataset. Under Request settings, enter a name and optional description for the request, followed by Submit.

A completed dataset expiration request with the Request settings and Submit button highlighted.

A Confirm request dialog appears. You are asked to confirm the dataset name and the date that the dataset will be deleted by. Select Submit to continue.

After the request is submitted, a work order is created and appears on the main tab of the Data Lifecycle workspace. From here, you can monitor the work order’s status as it processes the request.

NOTE
Refer to the overview section on timelines and transparency for details on how dataset expirations are processed once they are executed.

Edit or cancel a dataset expiration edit-or-cancel

To edit or cancel a dataset expiration, select Dataset on the main page of the workspace, and select the dataset expiration from the list.

On the details page of the dataset expiration, the right rail shows controls to edit or cancel the scheduled deletion.

Next steps

This document covered how to schedule dataset expirations in the Experience Platform UI. For information on how to perform other data minimization tasks in the UI, refer to the data lifecycle UI overview.

To learn how to schedule dataset expirations using the data hygiene API, refer to the dataset expiration endpoint guide.

recommendation-more-help
332f81c1-51e7-4bde-8327-2eb07f09604f