Data Freshness and Record Lag: How to Avoid Treating Old Activity as a New Signal

6 min read · SiftSpot Insights

“New to the feed” is not the same as “new in the world.”

That is one of the most important rules in public-record monitoring.

A file downloaded today can contain events from months earlier. A status page updated recently can still describe a license first issued years ago. A corporate record can show both a filing date and a different effective date. A government data product can be released on a schedule that lags the activity it measures.

If those dates are collapsed into one “date” field, stale activity can look fresh.

The solution is not complicated: preserve the timeline the source gives you.

One record can have many legitimate dates

Public systems use dates for different administrative purposes.

Sunbiz corporate detail records can include Date Filed, Effective Date, Event Date Filed, Event Effective Date, and annual-report filing dates. [S2]

DBPR alcoholic-beverage files include Original Issue Date, Effective Date, and Expiration Date. [S3]

DBPR restaurant and lodging change extracts can include application approval dates in addition to license and inspection information. [S4] [S5]

Census Business Formation Statistics distinguish business applications from actual or projected formations and publish products on different release schedules and geographic frequencies. [S7] [S8]

None of those dates should automatically replace the others.

A freshness model should name the date before using it.

Event date

The event date is the date most closely tied to the business change the workflow cares about.

Depending on the source, that might be an approval date, effective date, filing date, inspection date, or another defined field.

The important part is consistency. If a watchlist ranks owner-change records by application approval date, use that definition for all comparable records unless the source changes.

Do not silently switch to discovery date when the event date is inconvenient or missing.

Filing date

A filing date tells you when a document entered a filing system.

That can be the most appropriate event date for some research questions, but not all.

A filing can have an effective date that differs from the date filed. A later annual report can update officers without changing the original formation date. A recent filed document can refer to an entity that has existed for years.

The filing date is therefore a fact about the filing, not a universal measure of business age or operational change.

Effective or approval date

Effective and approval dates can be more useful for regulated activity because they may describe when a credential or application took effect.

But those fields still need context.

An alcoholic-beverage license’s effective date is not the same as its original issue date. A restaurant owner-change approval date is not automatically the date the public-facing business changed hands. [S3] [S4]

Use the date for what the source says it means.

Publication or extract date

A downloadable public-record file has its own freshness.

The extract may have been generated recently while containing older events. DBPR’s current-fiscal-year new-establishment and owner-change files are an example of why this matters: a “current” file can contain all qualifying approvals from the current fiscal period, not only activity from the day the analyst downloaded it. [S4]

The file’s freshness tells you when the dataset was observed or published. It does not reset the age of every event inside it.

Discovery date

Discovery date belongs to the research process, not the underlying business event.

It answers:

When did our system or analyst first see this record?

That field is operationally useful. It helps audit monitoring coverage, identify missed updates, and measure research backlog.

It should never be allowed to masquerade as the event date.

If an owner-change approval occurred weeks ago but entered the queue today, keep both facts:

Event date: when the approval occurred according to the source.

Discovery date: when the research system found it.

That distinction prevents the phrase “new signal” from becoming misleading.

Record lag is normal

Public data systems are not synchronized in real time.

One agency may update a searchable record quickly. Another may publish a periodic extract. A separate statistical product may intentionally release data after a defined lag. A business website may change before or after government records reflect the same transition.

Lag is not automatically an error. It is a property of the source.

A strong source registry should therefore document:

• how often the source is normally checked;
• which date fields it exposes;
• whether it is a live search, periodic extract, historical file, or statistical release;
• what field the workflow uses for event recency;
• what lag or backfill behavior the reviewer should expect.

That turns freshness from a guess into a documented part of the methodology.

Watch for the four common freshness traps

Trap 1: the recently updated page

A page modification date can change because the site layout or file was refreshed. That does not mean every underlying business record is new.

Trap 2: the newly downloaded historical event

A current extract can contain earlier events. Rank by the event field, not the download time.

Trap 3: the old original issue date beside a new status

A license can be old while its status is current. Preserve both. Do not label the entire record stale just because the original credential is old, and do not label the business new just because one status changed.

Trap 4: the first-seen illusion

A monitoring system that starts today will discover many existing records for the first time. Those are baseline records, not automatically new business changes.

Initial ingestion should be clearly separated from subsequent change detection.

Use a freshness tuple instead of one date

A compact record can store four fields:

1. event_date — the source field used to represent the change;

2. source_as_of — when the source or extract was current, if known;

3. discovered_at — when the research system found it;

4. verified_at — when a human or workflow last confirmed the source.

Not every source will provide all four. Missing fields should remain missing rather than being inferred.

This small structure answers several important questions immediately:

• Is the event itself recent?
• Is the source view current?
• Did the monitoring process discover the event late?
• Has anyone rechecked the record since it entered the queue?

Define a stale-event rule

A research queue needs an explicit rule for when an event is too old to deserve normal priority.

The rule can vary by event type.

A new-establishment approval may lose value quickly if the location has been open for months. An ownership change may still matter later if identity has not been resolved. A corporate filing may be useful as background even when it is not a current prospecting signal.

The important part is that the rule measures research value, not presumed insurance intent.

A stale item can still be historically useful. It simply should not be presented as a fresh trigger.

Preserve supersession

Freshness also means recognizing when a newer authoritative record replaces an older one.

If a later filing lists different managers, do not overwrite the old filing and forget it. Mark the older record as historical and preserve the newer one as current.

If a license changes status, preserve the prior observation and the current status with their dates.

If an owner-change extract is followed by later licensing information that clarifies the operator, connect the records rather than treating them as unrelated discoveries.

The history is often the signal.

The practical freshness test

Before calling any record “new,” ask:

• What exactly is new—the event, the filing, the status, the extract, or our awareness of it?
• Which source field carries that meaning?
• Is there a later record that supersedes it?
• Is the event inside the workflow’s review window?
• Can another reviewer reproduce the timeline from the stored source notes?

If those answers are visible, the queue can use recency without exaggeration.

Fresh data is useful. Correctly dated data is more useful.