Key Features:
- Real-time data delivery: Lowest latency option for streaming Posts as they’re published.
- Precise filtering: Filter for exactly the data you are looking for using Boolean queries with operators.
- Delivery: JSON response over HTTP/1.1 chunked transfer encoding.
- Local datacenter support: Fetch Posts only from the local datacenter to further reduce latency by avoiding replication lag.
The Powerstream API is a premium offering available under select Enterprise plans.If you’re interested in accessing Powerstream or learning more about our Enterprise offerings, please reach out to our Sales team by submitting the Enterprise Request Form.
We’ll be happy to discuss how Powerstream can support your needs.
Quick Start
This section showcases how to quickly get started with the PowerStream endpoints using Python with therequests library. Install it via pip install requests. All examples use OAuth 2.0 Bearer Token authentication. Replace YOUR_BEARER_TOKEN with your actual token (store it securely, e.g., via os.getenv('BEARER_TOKEN')).
We’ll cover each endpoint with code snippets. Assume these imports at the top:
Setup
1. Create Rules (POST /rules)
Add rules to filter your stream.2. Delete Rules (POST /rules)
Remove rules by ID (recommended) or value.3. Get Rules (GET /rules)
Fetch all active rules.4. PowerStream (GET /stream)
Connect to the stream for real-time, low-latency Posts. Usestream=True for line-by-line reading. Implement reconnect logic for robustness.
Local Datacenter Support
For latency optimization, Powerstream provides an option to fetch only posts that originated or were created in the local datacenter where a connection is established. This avoids replication lag, resulting in faster delivery compared to posts from other datacenters. To enable this, append the query parameter?localDcOnly=true to the stream endpoint (e.g., /2/powerstream?localDcOnly=true). The datacenter you are connected to will be indicated both in the initial data payload of the stream and as an HTTP header in the response.
To use in code:
localDcOnly parameter is enabled, when the stream first connects, it will include the following response headers indicating which local datacenter is being used:
Tip: To optimize latency, set up connections from different geographic locations (e.g., one near Atlanta on the US East Coast and another near Portland on the US West Coast), enabling
localDcOnly=true for each. This provides faster access to posts from each respective datacenter. Aggregate the streams on your end to combine cross-datacenter data.Operators
In order to set rules for filtering, you can use keywords and operators.Powerstream operators
Complete list of available operators
Responses
The payload of the Powestream API is the same format as the legacy GNIP Powertrack API. A sample json response looks like:Limits & Best Practices
- Rate Limits: 50 requests/24h for rule management; no limit on streams (but connection limits apply).
- Reconnection: Exponential backoff on disconnects.
- Monitoring: Use
Connection: keep-aliveheaders.