Skip to content

feat(integrations): hosted email-enrichment providers + cascade wiring#5087

Merged
TheodoreSpeaks merged 9 commits into
stagingfrom
feat/enrichment-providers
Jun 16, 2026
Merged

feat(integrations): hosted email-enrichment providers + cascade wiring#5087
TheodoreSpeaks merged 9 commits into
stagingfrom
feat/enrichment-providers

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Summary

  • Add Datagma, Dropcontact, LeadMagic, Icypeas, and Enrow integrations — tools, blocks, brand icons, and BYOK + metered hosted-key support
  • Wire the new finders/verifiers into all five enrichment cascades: work-email, phone-number, email-verification, company-info, company-domain
  • Add hosting tests for all five providers plus cascade tests (new test files for email-verification, company-info, company-domain)

Type of Change

  • New feature

Testing

Tested manually. bun run lint, bun run check:api-validation:strict, and tsc --noEmit all pass; 161 unit tests pass (hosting + cascade + blocks). Live-API verification of per-credit pricing and provider response shapes still pending.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Add Datagma, Dropcontact, LeadMagic, Icypeas, and Enrow integrations —
tools, blocks, brand icons, and BYOK + metered hosted-key support — and
register each in the tool/block registries and BYOK provider list.

Wire the new finders/verifiers into the enrichment cascades:
- work-email: Datagma, LeadMagic, Dropcontact, Icypeas, Enrow
- phone-number: LeadMagic, Datagma, Dropcontact
- email-verification: Icypeas, Enrow
- company-info: Datagma, LeadMagic
- company-domain: Datagma

Add hosting tests for all five providers and cascade tests covering the
new providers (incl. new test files for email-verification, company-info,
and company-domain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 16, 2026 10:57pm

Request Review

@cursor

cursor Bot commented Jun 16, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Large surface area (external APIs, async polling, credit-based hosted billing) and enrichment waterfall behavior changes; mitigated by extensive unit tests but live API/pricing validation is still noted as pending.

Overview
Adds Datagma, Dropcontact, LeadMagic, Icypeas, and Enrow as first-class workflow integrations: HTTP tools (including async polling for Dropcontact, Enrow, and Icypeas), canvas blocks, brand icons, BYOK entries in workspace settings, and metered hosted-key billing via per-provider credit→USD helpers.

Those providers are appended to the existing enrichment waterfalls—work email, phone, email verification, company info, and company domain—each with buildParams / mapOutput adapters (e.g. Enrow work-email only accepts valid qualifications; Icypeas supports mononym find-email rows). Hosting and cascade unit tests cover pricing rules and provider ordering.

Reviewed by Cursor Bugbot for commit 68af663. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread apps/sim/tools/icypeas/verify_email.ts
Comment thread apps/sim/tools/icypeas/verify_email.ts
@greptile-apps

greptile-apps Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds five new email-enrichment integrations — Datagma, Dropcontact, LeadMagic, Icypeas, and Enrow — each with tool definitions, block UI, brand icons, and full BYOK + hosted-key support, then wires all five into the five enrichment cascades (work-email, phone-number, email-verification, company-info, company-domain).

  • New provider tools (5 × multiple endpoints): Datagma (find-email, find-phone, enrich-company, enrich-person, get-credits), Dropcontact (enrich-contact async poll), LeadMagic (find-email, find-mobile, company-search, profile-search, validate-email, etc.), Icypeas (find-email async poll, verify-email async poll), and Enrow (find-email async poll, verify-email async poll). Each carries per-endpoint credit billing, a shared hosting config, and matching BYOK provider IDs.
  • Cascade extensions: All five providers are appended to work-email; LeadMagic and Datagma extend phone-number; Icypeas and Enrow extend email-verification; Datagma and LeadMagic extend company-info; Datagma extends company-domain.
  • Previously reported cascade bugs fixed: Both icypeas_find_email and icypeas_verify_email now correctly return { success: true } for all terminal statuses (NOT_FOUND, DEBITED_NOT_FOUND, BAD_INPUT, etc.), matching the cascade runner contract.

Confidence Score: 5/5

This PR is safe to merge. All cascade integrations follow the established contract, and the previously reported Icypeas success/false bugs are correctly resolved.

The Icypeas async-poll tools now return {success:true} for every terminal status, closing the cascade-runner mismatch reported in review. Cascade wiring for all five new providers is correct: guards, mapOutput logic, and billing functions all behave as intended. The only open items are two non-blocking observations — the Datagma tools' success:false pattern creates a latent mismatch with the 404-detection path (harmless today because Datagma returns HTTP 200 for not-found), and the company-domain Datagma step uses a 2-credit full-profile endpoint just to retrieve a domain.

All Datagma tool files (datagma/find_email.ts, find_phone.ts, enrich_company.ts, enrich_person.ts) use success:false returns on HTTP errors while every other new provider throws; this is inconsistent and could mask a 404 not-found case as an infrastructure error if the Datagma API ever changes its not-found status code.

Important Files Changed

Filename Overview
apps/sim/enrichments/work-email/work-email.ts Appends Datagma, LeadMagic, Dropcontact, Icypeas, and Enrow providers to the waterfall; all buildParams guards are correct, filterUndefined is imported and used consistently, and Enrow's quality-filter (qualification === 'valid') correctly avoids placing unvalidated addresses into the cell.
apps/sim/tools/icypeas/find_email.ts Async poll tool; postProcess now correctly returns {success:true} for all terminal statuses (NOT_FOUND, BAD_INPUT, INSUFFICIENT_FUNDS, ABORTED), fixing the previously reported cascade-runner mismatch.
apps/sim/tools/icypeas/verify_email.ts Async poll tool with the same terminal-status fix as find_email; billing getCost still throws when output.status is falsy (reported in previous thread). All postProcess paths return success:true.
apps/sim/tools/dropcontact/enrich_contact.ts Async submit-then-poll pattern; correctly distinguishes pending ({success:false,error:false}), error ({error:true}), and ready ({success:true}) poll states. Billing charges only when email_found is true.
apps/sim/tools/datagma/find_email.ts Synchronous tool; transformResponse returns {success:false} for HTTP errors (inconsistent with other new tools that throw, but functionally equivalent via cascade catch). API key placed in URL query param per Datagma's documented scheme (noted in previous thread).
apps/sim/tools/enrow/find_email.ts Async submit-poll (202/200 pattern); poll throws on non-2xx, times out after 120 s, and correctly returns success:true on completion. Billing charges only for qualification==='valid' results.
apps/sim/enrichments/email-verification/email-verification.ts Adds Icypeas and Enrow to the verifier waterfall; mapOutput correctly maps FOUND/DEBITED→valid and NOT_FOUND/DEBITED_NOT_FOUND→invalid for Icypeas, and qualification==='valid'/'invalid' for Enrow; unknown/inconclusive results return null to fall through.
apps/sim/enrichments/company-info/company-info.ts Adds Datagma (companySize/shortDescription) and LeadMagic (employeeRange
apps/sim/lib/api/contracts/byok-keys.ts Adds datagma, dropcontact, leadmagic, icypeas, enrow to the byokProviderIdSchema enum; change is additive and backward-compatible.
apps/sim/app/workspace/[workspaceId]/settings/components/byok/byok.tsx New provider entries added to both PROVIDERS and PROVIDER_SECTIONS arrays; icons, descriptions, and placeholders are consistent with existing entries.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant CascadeRunner as enrichments/run.ts
    participant Hunter
    participant Findymail
    participant Prospeo
    participant Wiza
    participant PDL
    participant Datagma
    participant LeadMagic
    participant Dropcontact
    participant Icypeas
    participant Enrow

    Caller->>CascadeRunner: runEnrichment(work-email, inputs)
    CascadeRunner->>Hunter: executeTool(hunter_email_finder)
    Hunter-->>CascadeRunner: "{success:true, output:{email:null}}"
    Note over CascadeRunner: mapOutput→null, fall through
    CascadeRunner->>Findymail: executeTool(findymail_find_email_from_name)
    Findymail-->>CascadeRunner: "{success:true, output:{contact:{email:"..."}}}"
    alt email found
        CascadeRunner-->>Caller: "{result:{email}, provider:"Findymail"}"
    else no result
        CascadeRunner->>Prospeo: executeTool(...)
        CascadeRunner->>Wiza: executeTool(...)
        CascadeRunner->>PDL: executeTool(...)
        CascadeRunner->>Datagma: executeTool(datagma_find_email)
        Note over Datagma: GET ?apiId=key (URL param auth)
        CascadeRunner->>LeadMagic: executeTool(leadmagic_find_email)
        CascadeRunner->>Dropcontact: executeTool(dropcontact_enrich_contact)
        Note over Dropcontact: POST submit → poll until ready
        CascadeRunner->>Icypeas: executeTool(icypeas_find_email)
        Note over Icypeas: POST submit → poll until terminal status
        CascadeRunner->>Enrow: executeTool(enrow_find_email)
        Note over Enrow: POST submit → poll until 200
        Enrow-->>CascadeRunner: "{success:true, output:{email, qualification:"valid"}}"
        CascadeRunner-->>Caller: "{result:{email}, provider:"Enrow"}"
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant CascadeRunner as enrichments/run.ts
    participant Hunter
    participant Findymail
    participant Prospeo
    participant Wiza
    participant PDL
    participant Datagma
    participant LeadMagic
    participant Dropcontact
    participant Icypeas
    participant Enrow

    Caller->>CascadeRunner: runEnrichment(work-email, inputs)
    CascadeRunner->>Hunter: executeTool(hunter_email_finder)
    Hunter-->>CascadeRunner: "{success:true, output:{email:null}}"
    Note over CascadeRunner: mapOutput→null, fall through
    CascadeRunner->>Findymail: executeTool(findymail_find_email_from_name)
    Findymail-->>CascadeRunner: "{success:true, output:{contact:{email:"..."}}}"
    alt email found
        CascadeRunner-->>Caller: "{result:{email}, provider:"Findymail"}"
    else no result
        CascadeRunner->>Prospeo: executeTool(...)
        CascadeRunner->>Wiza: executeTool(...)
        CascadeRunner->>PDL: executeTool(...)
        CascadeRunner->>Datagma: executeTool(datagma_find_email)
        Note over Datagma: GET ?apiId=key (URL param auth)
        CascadeRunner->>LeadMagic: executeTool(leadmagic_find_email)
        CascadeRunner->>Dropcontact: executeTool(dropcontact_enrich_contact)
        Note over Dropcontact: POST submit → poll until ready
        CascadeRunner->>Icypeas: executeTool(icypeas_find_email)
        Note over Icypeas: POST submit → poll until terminal status
        CascadeRunner->>Enrow: executeTool(enrow_find_email)
        Note over Enrow: POST submit → poll until 200
        Enrow-->>CascadeRunner: "{success:true, output:{email, qualification:"valid"}}"
        CascadeRunner-->>Caller: "{result:{email}, provider:"Enrow"}"
    end
Loading

Reviews (6): Last reviewed commit: "fix(enrichment): only accept valid-quali..." | Re-trigger Greptile

Comment thread apps/sim/tools/icypeas/verify_email.ts
Comment thread apps/sim/tools/icypeas/find_email.ts
Comment on lines +65 to +73
const status = output.status as string | undefined
if (!status) {
throw new Error('Icypeas verify-email: cannot determine cost — status is missing')
}
// Billable when the status name contains DEBITED (i.e. DEBITED or DEBITED_NOT_FOUND).
const billable = status.includes('DEBITED')
// 0.1 credit; express as a fractional number so ICYPEAS_CREDIT_USD math works.
return billable ? 0.1 : 0
}),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Billing function throws when status is falsy

getCost throws 'cannot determine cost — status is missing' if output.status is falsy. In normal operation postProcess always returns a terminal status, so this is unreachable in practice. But if the hosting layer ever evaluates cost on the initial transformResponse result (before postProcess runs), output.status will be null and the throw propagates as an unhandled billing error. Consider returning 0 when status is absent, matching the defensive posture of the other providers.

Comment on lines +57 to +68
type: 'string',
required: true,
visibility: 'user-only',
description: 'Datagma API key',
},
},

request: {
url: (params) => {
const url = new URL('https://gateway.datagma.net/api/ingress/v6/findEmail')
url.searchParams.set('apiId', params.apiKey)
url.searchParams.set('fullName', params.fullName)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 API key embedded in URL query string (?apiId=...) for all Datagma endpoints

Every Datagma tool appends url.searchParams.set('apiId', params.apiKey). This is Datagma's documented auth scheme and can't be changed client-side, but the hosted API key appears verbatim in every request URL and will be captured by server-side access logs at Datagma and any intermediary. A note in the DATAGMA_API_KEY_PREFIX doc comment that this API uses URL-parameter auth would help operators understand the risk when rotating a compromised key.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment thread apps/sim/tools/icypeas/find_email.ts
Comment thread apps/sim/tools/datagma/enrich_person.ts
- Icypeas find_email/verify_email postProcess return success:true for all
  terminal statuses (NOT_FOUND/DEBITED_NOT_FOUND included) so the cascade
  runner calls mapOutput and records invalid/not-found verdicts instead of
  throwing and inflating the error count
- Bill Icypeas verify FOUND (not just DEBITED*) per the documented 0.1-credit
  charge
- Datagma enrich_person only applies the 30-credit phone surcharge when a
  phone lookup (phoneFull) was requested
- Note Datagma's URL-param (apiId) auth in the hosted-key doc comment
- Update hosting tests to match

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread apps/sim/tools/enrow/verify_email.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

getCost returned a flat 0.25 credits regardless of output, so a job that
fell back to the initial submit response (poll never completed, no
qualification) was still metered. Charge 0.25 only when a qualification is
present; 0 otherwise. Add a no-qualification test case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread apps/sim/enrichments/work-email/work-email.ts Outdated
Comment thread apps/sim/enrichments/work-email/work-email.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

…aid plan

Align *_CREDIT_USD to the entry tier Sim will provision:
- Datagma: Regular $49/3,000 emails → $0.0163 (was Popular $0.0132)
- LeadMagic: Basic $49/2,000 → $0.0245 (was Growth $0.0104)

Icypeas (Basic $0.019), Enrow (Starter $0.012), and Dropcontact (Starter
~$0.17) already reflect their lowest plan. Tests derive from the constants,
so values stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
}
}

throw new Error('Dropcontact enrichment did not complete within the polling window')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Poll failures look like misses

Medium Severity

When async postProcess polling times out or hits poll errors, these tools throw, but executeTool catches that and keeps the initial stub with success: true. Enrichment then treats timeouts and poll failures as a clean no-match instead of an error, and may keep trying later providers.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit b9aae9d. Configure here.

Comment thread apps/sim/tools/icypeas/verify_email.ts
… map

- work-email LeadMagic: pass full_name + domain so single-token (mononym)
  names are no longer skipped
- work-email Icypeas: firstname/lastname are optional on the API, so run a
  mononym with firstname alone instead of self-skipping
- icypeas_verify_email mapItem reads item.email (verify payload shape) with a
  fallback to the nested results.emails[0].email

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread apps/sim/tools/enrow/find_email.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

getCost compared qualification to exactly 'valid' while the cascade
normalizes with toLowerCase(), so a differently-cased API qualifier could
zero out billing on a valid email. Lowercase before comparing; add a test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 33dc8dc. Configure here.

Comment thread apps/sim/enrichments/work-email/work-email.ts
Dropcontact is an email/company-data enrichment service, not a phone-discovery
provider — its phone/mobile_phone fields are unreliable and were surfacing
firmographic data (an employee-count range like "5000-20077") as the phone.
Keep the two purpose-built phone finders (LeadMagic find_mobile, Datagma
find_phone); Dropcontact stays in the work-email and company cascades where
its data is reliable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enrow's finder qualifies each email valid/invalid. The work-email mapOutput
accepted any non-empty email, so an invalid-qualified address could fill the
cell while hosted billing (which only charges on valid) charged zero. Gate the
cell on qualification === 'valid', consistent with billing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@TheodoreSpeaks TheodoreSpeaks merged commit 15a970d into staging Jun 16, 2026
16 checks passed
@TheodoreSpeaks TheodoreSpeaks deleted the feat/enrichment-providers branch June 16, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant