Integrations/pull contact data by leogermani · Pull Request #4496 · Automattic/newspack-plugin

leogermani · 2026-02-18T21:55:24Z

All Submissions:

Have you followed the Newspack Contributing guideline?
Does your code follow the WordPress' coding standards and VIP Go coding standards?
Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Changes proposed in this Pull Request:

Implements the pull contact data strategy.

It triggers an async pull every five minutes when the user is active in the site.

If the user is returning after some time away:

It will trigger an immediate async pull if they were around in the last 24h
it will trigger a synchronous pull if last time they were around was over 24h ago

The idea is that we want to get fresh data on their first request to be able to properly segment them.

But, we time cap this - the sync request should not hold the page load for too long.

The goal here is to make sure the user is properly segmented on the first page load when they are back after some time away.

TODOs:

We might need to change how we store selected fields when we have more than just keys there (data types, etc)
Need to surface pull errors. Wonder if we can combine those with other errors in the dedicated view we are going to build. How to store them?

How to test the changes in this Pull Request:

Open WP Shell and set a selected field for the ESP integration. Ex:

$i = new Newspack\Reader_Activation\Integrations\ESP();
$i->set_selected_fields( [ 'HAS_SOMETHING' ] );

Add a value to this metadata in a contact directly in your ESP dashboard
Visit the site logged in as that reader. Look for something like this in the logs:

[NEWSPACK][Newspack\Reader_Activation\Integrations\Contact_Pull::pull_single_integration]: Pulled data from esp: {"HAS_SOMETHING":"yes"}
[NEWSPACK][Newspack\Reader_Activation\Integrations\Contact_Pull::pull_sync]: Loopback pull succeeded for esp.

Wait 5 minutes - or edit the code to make the interval shorter - and watch it again in the logs
See the actions piling up with: wp action-scheduler action list --hook=newspack_pull_integration_contact_data
Verify that the metadata value was stored as user meta:

newspack_reader_data_item_HAS_SOMETHING                         | yes

Clear the np_integrations_last_pull metadata for the user and add a sleep in the integration callback (or lower the timeout limit).
Confirm you see the error in the log and that an action is scheduled right away (all should be printed in the logs)

Other information:

Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your changes, as applicable?
Have you successfully ran tests with your changes locally?

Copilot

Pull request overview

This PR implements a pull contact data strategy that periodically fetches contact metadata from integrations (e.g., ESPs) and stores it in WordPress user meta. The implementation includes throttling logic to run synchronous pulls when data is stale (>24 hours) and async pulls for fresh data (<24 hours but >5 minutes old), with a 5-second time limit on synchronous operations to avoid blocking page loads.

Changes:

Adds Contact_Pull orchestration class to manage sync/async pulling with configurable intervals and time limits
Extends Integration base class with pull_contact_data() abstract method and selected fields storage/retrieval
Implements ESP integration's pull_contact_data() using Newspack Newsletters API

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
includes/reader-activation/integrations/class-contact-pull.php	New class implementing pull orchestration with sync/async logic, time limits, and Action Scheduler integration
includes/reader-activation/integrations/class-integration.php	Adds abstract pull_contact_data() method and get/set_selected_fields() for field selection
includes/reader-activation/integrations/class-esp.php	Implements pull_contact_data() to fetch metadata from ESP via Newspack Newsletters
includes/reader-activation/class-integrations.php	Initializes Contact_Pull hooks and includes new class file
tests/unit-tests/integrations/class-test-integrations.php	Comprehensive test coverage for pull functionality including throttling, sync/async behavior, error handling, and field filtering
tests/unit-tests/integrations/class-sample-integration.php	Adds pull_contact_data() stub to test integration
tests/unit-tests/integrations/class-failing-sample-integration.php	Adds pull_contact_data() and get_incoming_available_contact_fields() stubs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

includes/reader-activation/integrations/class-esp.php

includes/reader-activation/integrations/class-integration.php

includes/reader-activation/integrations/class-contact-pull.php

includes/reader-activation/integrations/class-integration.php

miguelpeixe

I still haven't gone through testing everything, but I wanted to respond to the sync pull strategy, since it's an important aspect of this.

includes/reader-activation/integrations/class-contact-pull.php

miguelpeixe · 2026-02-24T20:07:35Z

includes/reader-activation/integrations/class-contact-pull.php

+	/**
+	 * Max seconds for the entire synchronous pull routine.
+	 *
+	 * @var int
+	 */
+	const PULL_TIME_LIMIT = 5;


This is a lot of seconds to let a reader hanging. I suggest something around 700ms or less

I tried 1 but we have to consider that there's the overhead of loading WP... At least in my local environment, not even 2 seconds was enough. I'll leave it at 2 for now.

See my top level comment about yet having to validate if this is gonna be useful or not. Then we can revisit

Can we place this behind a filter? Our local envs are notoriously not the fastest.

yes, good call

includes/reader-activation/integrations/class-contact-pull.php

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

leogermani · 2026-02-25T20:20:02Z

@miguelpeixe thanks for pointing me to the obvious!

Looks good now, but there's still an open question we might be able to answer later in the project:

If we update the Reader data at that point, will the data be available in time to be used by the content gate checks, and the popups segmentation?

In a quick test I did, the data added is not available in the the_post hook yet, we probably need to reload all usermeta for that user, because the update happened in another request. Need to look a bit deeper into this.

Also, I don't see the data being hydrated into the browser local storage... which I think is expected (and I suspect I saw happening the other day...)

miguelpeixe · 2026-02-26T15:59:48Z

In a quick test I did, the data added is not available in the the_post hook yet, we probably need to reload all usermeta for that user, because the update happened in another request. Need to look a bit deeper into this.

Because it executed in a separate PHP process, it's likely that the data was already fetched from the object cache, so it's already in memory and need to be flushed/refetched. Maybe wp_cache_flush_runtime()?

…-data

leogermani · 2026-02-27T18:34:45Z

In a quick test I did, the data added is not available in the the_post hook yet, we probably need to reload all usermeta for that user, because the update happened in another request. Need to look a bit deeper into this.

Because it executed in a separate PHP process, it's likely that the data was already fetched from the object cache, so it's already in memory and need to be flushed/refetched. Maybe wp_cache_flush_runtime()?

We can look into it later when we start using this data for segmentation and access control

…-data

includes/reader-activation/integrations/class-contact-pull.php

miguelpeixe · 2026-03-03T12:38:19Z

includes/reader-activation/integrations/class-esp.php

+			return new \WP_Error( 'user_not_found', __( 'User not found.', 'newspack-plugin' ) );
+		}
+
+		$contact_data = Newspack_Newsletters_Subscription::get_contact_data( $user->user_email, true );


Could use a class_exists here to avoid a fatal error. This could live under can_sync() for broader coverage.

it is behind can_sync already

Ugh, morning brain. Sorry about that

miguelpeixe · 2026-03-03T12:42:12Z

includes/reader-activation/integrations/class-integration.php

+	 *
+	 * @var string
+	 */
+	const OPTION_PREFIX = 'np_integration_selected_fields_';


We usually go for newspack_ as the prefix. The database limit is 191 characters, it should hold fine.

miguelpeixe · 2026-03-03T12:42:35Z

includes/reader-activation/integrations/class-contact-pull.php

+	 *
+	 * @var string
+	 */
+	const LAST_PULL_META = 'np_integrations_last_pull';


Same here, newspack_ as the prefix

miguelpeixe · 2026-03-03T12:43:49Z

includes/reader-activation/integrations/class-esp.php

+		if ( ! $this->can_sync() ) {
+			return new \WP_Error( 'missing_dependency', __( 'ESP Integration is not fully configured.', 'newspack-plugin' ) );
+		}


We can benefit from better error handling by setting $return_errors = true in can_sync() here.

Co-authored-by: Miguel Peixe <miguel.peixe@automattic.com>

miguelpeixe

Need to update the tests

leogermani added 4 commits February 18, 2026 18:28

feat(integrations): store selected pull fields

d09e07f

feat(integrations): basic pulling strategy

84a770a

feat(integrations): pull async and refactor class

ff87026

feat(integrations): implement ESP pull

570bdbb

leogermani marked this pull request as ready for review February 19, 2026 14:55

leogermani requested a review from a team as a code owner February 19, 2026 14:55

leogermani requested a review from Copilot February 19, 2026 14:55

Copilot started reviewing on behalf of leogermani February 19, 2026 14:55 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

leogermani self-assigned this Feb 19, 2026

leogermani added the [Status] Needs Review The issue or pull request needs to be reviewed label Feb 19, 2026

miguelpeixe requested changes Feb 24, 2026

View reviewed changes

github-actions bot added the [Status] Needs Changes or Feedback The issue or pull request needs action from the original creator label Feb 24, 2026

leogermani and others added 2 commits February 25, 2026 17:05

feat: better approach for integration pull requests

aef95ae

Update includes/reader-activation/integrations/class-contact-pull.php

87cec18

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

leogermani requested a review from miguelpeixe February 25, 2026 20:20

leogermani added 3 commits February 27, 2026 12:27

Merge branch 'integrations/get-fields' into integrations/pull-contact…

c5c7b14

…-data

chore: make pulling data optional for integrations

3f8d172

feat(integrations): add the timeout value behind a filter

1a7c538

miguelpeixe added 2 commits March 2, 2026 16:59

Merge branch 'trunk' into integrations/pull-contact-data

21a7862

Merge branch 'integrations/get-fields' into integrations/pull-contact…

5382372

…-data

miguelpeixe requested changes Mar 3, 2026

View reviewed changes

leogermani and others added 3 commits March 3, 2026 09:51

Update includes/reader-activation/integrations/class-contact-pull.php

1a2e614

Co-authored-by: Miguel Peixe <miguel.peixe@automattic.com>

chore: better option names

3e232a5

chore: better error handling

0ed1444

leogermani requested a review from miguelpeixe March 3, 2026 13:51

miguelpeixe requested changes Mar 3, 2026

View reviewed changes

test: fix tests

e1bca43

miguelpeixe approved these changes Mar 3, 2026

View reviewed changes

github-actions bot added [Status] Approved The pull request has been reviewed and is ready to merge and removed [Status] Needs Review The issue or pull request needs to be reviewed [Status] Needs Changes or Feedback The issue or pull request needs action from the original creator labels Mar 3, 2026

leogermani merged commit 3723cef into integrations/get-fields Mar 3, 2026
9 checks passed

leogermani deleted the integrations/pull-contact-data branch March 3, 2026 16:25

Conversation

leogermani commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

All Submissions:

Changes proposed in this Pull Request:

How to test the changes in this Pull Request:

Other information:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

miguelpeixe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miguelpeixe Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leogermani commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

miguelpeixe commented Feb 26, 2026

Uh oh!

leogermani commented Feb 27, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

miguelpeixe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

leogermani commented Feb 18, 2026 •

edited

Loading

miguelpeixe Feb 24, 2026 •

edited

Loading

leogermani commented Feb 25, 2026 •

edited

Loading