Social Media

Twitter Data Extractor

Vern's Twitter extractor imports a new user's social media data — authors and tweets — into your product through a single managed migration, handling Twitter's auth, pagination, and rate limits for you. It's built for teams onboarding customers off Twitter who want their data in your app from day one.

Start migration

Browse all extractors

Twitter

SaaS API (REST)

Managed auth, retries, and throttling

Normalized JSON, CSV, and DB-ready output

Cleaned JSON, CSV, and DB-ready output

Export coverage

Know exactly what Vern pulls before you migrate.

Objects Vern extracts

Vern extracts the full set of queryable Twitter objects (2 total), including: Authors and Tweets. All of these support incremental change tracking, so syncs pull only what changed.

Output schema

Normalized JSON per object with referential keys preserved, so the relationships between your user's records survive the import. Timestamps are returned in ISO 8601. Delivered as JSON or CSV, or loaded straight into your own application database or pushed to your API — whatever your product expects.

Where it can land

Your own application database (Postgres, MySQL), a direct push to your API or webhook, or JSON/CSV files, plus another social media system (Blogger, Buzzsprout, Instagram) when a user is switching tools.

Source details

Built for the source type shown above, with extractor content refreshed from the collection record.

Managed by Vern

The details you normally have to build yourself.

Authentication

API key / access token. Your user pastes their Twitter API key once when they sign up; Vern stores it encrypted and attaches it to every request, handling header signing and retries so you never touch the raw credential. See Twitter's authentication guide.

Pagination

Vern detects and handles Twitter's paging model (cursor, page-number, or offset) internally and returns one consistent, de-duplicated result set — so you import a user's complete data without ever managing page tokens.

Rate limits and gotchas

Twitter enforces its own API rate limits (see Twitter's rate-limit docs). Vern queues and throttles requests internally so importing a new user's data never trips a 429 — even on a large account.

API-key sources rarely surface their true rate ceilings until you hit a 429 mid-import, and many silently cap page size. Vern throttles to stay under the limit and normalizes paging so a partial pull never lands a user in your product with missing data.

API quickstart

1. Create a migration for Twitter
curl -X POST https://app.vern.so/api/v1/migrations \
  -H "x-api-key: $VERN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "Twitter → Postgres", "source": "Twitter" }'
→ { "migration_id": "mig_..." }
2. Generate the extraction recipe (pulls live via the Twitter API)
curl -X POST https://app.vern.so/api/v1/migrations/$MIGRATION_ID/runs \
  -H "x-api-key: $VERN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "kind": "generate", "extract_via_api": true }'
3. Run the approved extraction
curl -X POST https://app.vern.so/api/v1/migrations/$MIGRATION_ID/runs \
  -H "x-api-key: $VERN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "kind": "execute" }'
4. Download the normalized CSV export
curl https://app.vern.so/api/v1/migrations/$MIGRATION_ID/exports/{template} \
  -H "x-api-key: $VERN_API_KEY" -o twitter_export.csv

A cleaner way to extract without rebuilding source-specific plumbing.

View docs

Common questions

Can I keep a user's Twitter data in sync after the first import? Yes — 2 Twitter objects support incremental change tracking, so after the first full import Vern pulls only what changed on re-run.

Can I move a user from Twitter to Blogger? Yes — Vern normalizes the Twitter data model into typed tables that map cleanly onto Blogger and into your own product.

What does it cost to read the Twitter API? Twitter's own API access follows Twitter's pricing/plan; Vern is billed separately per import. Check Twitter's developer docs for their current limits.