Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Release

# Tag `vX.Y.Z` publishes facebook-scraper-api to PyPI with trusted publishing (no token).
#
# Once: pypi.org → Your account → Publishing → Add a pending publisher:
# PyPI project facebook-scraper-api, owner SocialAPIsHub,
# repository facebook-scraper-python, workflow release.yml, environment pypi.

on:
push:
tags:
- "v*.*.*"

concurrency:
group: release
cancel-in-progress: false

jobs:
pypi:
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/project/facebook-scraper-api/
permissions:
id-token: write
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: "3.12"
- run: pip install build
- run: python -m build
- name: Version matches tag
run: |
tag="${GITHUB_REF_NAME#v}"
version=$(python -c "import re;print(re.search(r'__version__ = \"(.+)\"', open('facebook_scraper_api/__init__.py').read()).group(1))")
test "$tag" = "$version" || { echo "::error::tag $tag != version $version"; exit 1; }
- uses: pypa/gh-action-pypi-publish@release/v1
28 changes: 28 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Test

on:
push:
branches: [main]
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- run: pip install -e ".[dev]"
- run: ruff check facebook_scraper_api tests
- run: ruff format --check facebook_scraper_api tests
- run: pytest -q tests
61 changes: 61 additions & 0 deletions PACKAGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# facebook-scraper-api

Public Facebook data from Python: Page details, Page and group posts, reels, videos, comments and search. Functions in the style of the old `facebook-scraper` (`get_posts(account, pages=...)`), but the data comes from a hosted API, so nothing breaks when Facebook changes its pages: **no login, no cookies, no browser, no proxies**.

Powered by [SocialAPIs](https://socialapis.io). **[Get a free API key](https://socialapis.io/auth/signup)**: 200 calls/month, no card.

```bash
pip install facebook-scraper-api
export SOCIALAPIS_TOKEN=your_key
```

```python
from facebook_scraper_api import get_page_info, get_posts, get_comments

page = get_page_info("nasa")
print(page["title"], page["followers_count"])

for post in get_posts("nasa", pages=2, limit=9): # 2 calls, 9 posts each
print(post)

for comment in get_comments("https://www.facebook.com/nasa/posts/...", pages=1):
print(comment)
```

Or call `set_api_token("...")` first instead of the environment variable.

| Function | What it returns |
|---|---|
| `get_page_info(account)` | A Page's details: name, followers, likes, category, contact, links |
| `get_posts(account, pages=1, limit=None, after=None, before=None)` | A Page's posts, newest first |
| `get_group_posts(group, pages=1, ...)` / `get_posts(group=...)` | A public group's posts |
| `get_group_info(group)` | A public group's details |
| `get_reels(account, pages=1)` / `get_videos(account, pages=1)` | A Page's reels / videos |
| `get_post(post_url, extended=False)` | One post's details |
| `get_comments(post_url, pages=1)` | Comments on a post or reel |
| `search_posts(query, pages=1)` / `search_pages(query, pages=1)` | Keyword search |

Listings are generators: they follow the API's `end_cursor` and stop after `pages` calls or when there are no more results. `account` is a Page URL or username; `group` is a group URL or id. For several keys or a custom connection, use `FacebookScraperAPI(api_token=...)`, which has the same methods.

## Command line

```bash
facebook-scraper-api page nasa
facebook-scraper-api posts nasa --pages 3 --limit 9 --out nasa.csv
facebook-scraper-api group-posts https://www.facebook.com/groups/123 --format jsonl
facebook-scraper-api comments https://www.facebook.com/nasa/posts/123 --pages 2
```

CSV columns are the fields of the records (nested fields as `reactions.total`).

## Costs

Most calls cost 1 credit. Page and group posts cost round(limit ÷ 3) credits per call (limit 3-9, default 3). Failed calls (4xx / 5xx) are not charged. [Pricing](https://socialapis.io/pricing).

## Links

- [Endpoint reference](https://socialapis.io/api-sources/facebook-page-posts) · [Free Facebook follower count tool](https://socialapis.io/tools/facebook-follower-count)
- Source and migration guide from `facebook-scraper`: [github.com/SocialAPIsHub/facebook-scraper-python](https://github.com/SocialAPIsHub/facebook-scraper-python)
- Full SDK (Instagram, Marketplace, Ads Library, async): [`socialapis-sdk`](https://pypi.org/project/socialapis-sdk/) · Meta Ads Library only: [`facebook-ads-library-api`](https://pypi.org/project/facebook-ads-library-api/)

Public data only. You are responsible for how you use it, including personal data rules (GDPR) where they apply. MIT license.
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,35 @@
# facebook-scraper-python

[![facebook-scraper-api on PyPI](https://img.shields.io/pypi/v/facebook-scraper-api?label=facebook-scraper-api&logo=pypi&logoColor=white)](https://pypi.org/project/facebook-scraper-api/)
[![PyPI](https://img.shields.io/pypi/v/socialapis-sdk?cacheSeconds=300&label=PyPI&logo=pypi&logoColor=white)](https://pypi.org/project/socialapis-sdk/)
[![Python versions](https://img.shields.io/pypi/pyversions/socialapis-sdk)](https://pypi.org/project/socialapis-sdk/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)

> Modern Python alternative to [`kevinzg/facebook-scraper`](https://github.com/kevinzg/facebook-scraper) — the 9.5k-star library that's been **abandoned since 2022**. Hosted backend means it doesn't break every time Meta changes their HTML.

This repo is a **migration landing page** + working examples. The actual SDK lives at [`SocialAPIsHub/socialapis-python`](https://github.com/SocialAPIsHub/socialapis-python) and ships as the `socialapis-sdk` package on PyPI.
This repo ships the **`facebook-scraper-api`** package: the same function names as `facebook-scraper` (`get_page_info`, `get_posts(account, pages=…)`, `get_group_info`, `get_comments`…), backed by a hosted API. It also has a command line tool.

```bash
pip install socialapis-sdk
pip install facebook-scraper-api
export SOCIALAPIS_TOKEN=your_key
```

```python
from socialapis import FacebookScraper
from facebook_scraper_api import get_page_info, get_posts

fb = FacebookScraper(api_token="...") # alias of Facebook — keeps your import line greppable
page = fb.get_page_info("EngenSA")
print(page.title, page.followers_count, page.category)
page = get_page_info("EngenSA")
print(page["title"], page["followers_count"])

for post in fb.get_page_posts("EngenSA").get("posts", []):
print(post["text"][:80])
for post in get_posts("EngenSA", pages=2): # follows the cursor, 1 call per page
print(post)
```

```bash
facebook-scraper-api posts EngenSA --pages 3 --limit 9 --out posts.csv
```

All functions, options and costs: [PACKAGE.md](PACKAGE.md). Need Instagram, Marketplace, the Ads Library or async too? The full SDK lives at [`SocialAPIsHub/socialapis-python`](https://github.com/SocialAPIsHub/socialapis-python) (`pip install socialapis-sdk`), and the rest of this page shows the migration with it.

**[Get a free API token →](https://socialapis.io/auth/signup)** — 200 calls/month, no credit card

---
Expand Down
224 changes: 224 additions & 0 deletions facebook_scraper_api/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
"""Facebook scraper API for Python: public Pages, posts, groups, reels, comments and search.

Works like the old ``facebook-scraper`` functions (``get_posts(account, pages=...)``)
but the data comes from a hosted API, so nothing breaks when Facebook changes
its pages: no login, no cookies, no browser, no proxies. Powered by
SocialAPIs (https://socialapis.io).

Quick start::

from facebook_scraper_api import get_page_info, get_posts, set_api_token

set_api_token("YOUR_API_TOKEN") # or set SOCIALAPIS_TOKEN

print(get_page_info("nasa")["followers_count"])
for post in get_posts("nasa", pages=2):
print(post)

Free API key (200 calls/month, no card): https://socialapis.io/auth/signup
"""

from __future__ import annotations

import os
from collections.abc import Callable, Iterator
from typing import Any

from socialapis import Facebook

from ._items import flatten, items_of, record_of
from ._paging import next_cursor, paginate

__version__ = "0.1.0"

__all__ = [
"FacebookScraperAPI",
"__version__",
"flatten",
"get_comments",
"get_group_info",
"get_group_posts",
"get_page_info",
"get_post",
"get_posts",
"get_reels",
"get_videos",
"items_of",
"next_cursor",
"search_pages",
"search_posts",
"set_api_token",
]


class FacebookScraperAPI:
"""All calls of this package on one client (one API key, one connection pool).

Every API call costs credits (1 per call for most; Page / group posts
cost round(limit / 3)). ``pages`` caps how many calls a listing makes.
"""

def __init__(self, api_token: str | None = None, *, client: Facebook | None = None) -> None:
if client is None:
token = api_token or os.environ.get("SOCIALAPIS_TOKEN")
if not token:
raise ValueError(
"Call set_api_token(...) or set SOCIALAPIS_TOKEN. Get a free key at "
"https://socialapis.io/auth/signup (200 calls/month, no card)."
)
client = Facebook(api_token=token)
self._fb = client

def __enter__(self) -> FacebookScraperAPI:
return self

def __exit__(self, *exc: object) -> None:
self.close()

def close(self) -> None:
self._fb.close()

def _items(self, call: Callable[..., dict[str, Any]], pages: int, **params: Any) -> Iterator[dict[str, Any]]:
for body in paginate(lambda cursor: call(end_cursor=cursor, **params), pages):
yield from items_of(body)

# ---- single records ----------------------------------------------------

def get_page_info(self, account: str, **params: Any) -> dict[str, Any]:
"""A Page's public details: name, followers, likes, category, contact, links."""
return self._fb.get_page_info(account, **params).model_dump(exclude_none=True)

def get_group_info(self, group: str, **params: Any) -> dict[str, Any]:
"""A public group's details: name, members, privacy, description."""
return self._fb.get_group_details(group, **params).model_dump(exclude_none=True)

def get_post(self, post_url: str, *, extended: bool = False, **params: Any) -> dict[str, Any]:
"""One post's details (``extended=True`` adds views, video URLs, music)."""
call = self._fb.get_post_details_extended if extended else self._fb.get_post_details
return record_of(call(post_url, **params))

# ---- listings ----------------------------------------------------------

def get_posts(
self,
account: str | None = None,
*,
group: str | None = None,
pages: int = 1,
limit: int | None = None,
after: str | None = None,
before: str | None = None,
**params: Any,
) -> Iterator[dict[str, Any]]:
"""Recent posts of a Page (``account``) or a public ``group``, newest first.

``limit`` is posts per call (3-9, default 3; cost round(limit / 3)
credits per call). ``after`` / ``before`` are ISO 8601 timestamps.
"""
if (account is None) == (group is None):
raise ValueError("give a Page (account) or a group, not both")
if limit is not None and not 3 <= limit <= 9:
raise ValueError("limit is posts per call, from 3 to 9")
target = account if account is not None else group
call = self._fb.get_page_posts if account is not None else self._fb.get_group_posts
yield from self._items(
lambda **kw: call(target, **kw), # type: ignore[arg-type]
pages,
limit=limit,
after_time=after,
before_time=before,
**params,
)

def get_reels(self, account: str, *, pages: int = 1, **params: Any) -> Iterator[dict[str, Any]]:
"""A Page's reels."""
yield from self._items(lambda **kw: self._fb.get_page_reels(account, **kw), pages, **params)

def get_videos(self, account: str, *, pages: int = 1, **params: Any) -> Iterator[dict[str, Any]]:
"""A Page's long-form videos."""
yield from self._items(lambda **kw: self._fb.get_page_videos(account, **kw), pages, **params)

def get_comments(self, post_url: str, *, pages: int = 1, **params: Any) -> Iterator[dict[str, Any]]:
"""Top-level comments on a post or reel."""
yield from self._items(lambda **kw: self._fb.get_post_comments(post_url, **kw), pages, **params)

def search_posts(self, query: str, *, pages: int = 1, **params: Any) -> Iterator[dict[str, Any]]:
"""Public posts matching a keyword."""
yield from self._items(lambda **kw: self._fb.search_posts(query, **kw), pages, **params)

def search_pages(self, query: str, *, pages: int = 1, **params: Any) -> Iterator[dict[str, Any]]:
"""Pages matching a keyword."""
yield from self._items(lambda **kw: self._fb.search_pages(query, **kw), pages, **params)


# ---------------------------------------------------------------------------
# Module-level functions, facebook-scraper style. They share one client,
# created on first use from set_api_token() or SOCIALAPIS_TOKEN.
# ---------------------------------------------------------------------------

_default: FacebookScraperAPI | None = None


def set_api_token(api_token: str) -> None:
"""Use this API key for the module-level functions."""
global _default
if _default is not None:
_default.close()
_default = FacebookScraperAPI(api_token)


def _client() -> FacebookScraperAPI:
global _default
if _default is None:
_default = FacebookScraperAPI()
return _default


def get_page_info(account: str, **params: Any) -> dict[str, Any]:
"""A Page's public details. See :meth:`FacebookScraperAPI.get_page_info`."""
return _client().get_page_info(account, **params)


def get_group_info(group: str, **params: Any) -> dict[str, Any]:
"""A public group's details. See :meth:`FacebookScraperAPI.get_group_info`."""
return _client().get_group_info(group, **params)


def get_post(post_url: str, **params: Any) -> dict[str, Any]:
"""One post's details. See :meth:`FacebookScraperAPI.get_post`."""
return _client().get_post(post_url, **params)


def get_posts(account: str | None = None, **params: Any) -> Iterator[dict[str, Any]]:
"""Posts of a Page or group. See :meth:`FacebookScraperAPI.get_posts`."""
return _client().get_posts(account, **params)


def get_group_posts(group: str, **params: Any) -> Iterator[dict[str, Any]]:
"""Posts of a public group."""
return _client().get_posts(group=group, **params)


def get_reels(account: str, **params: Any) -> Iterator[dict[str, Any]]:
"""A Page's reels."""
return _client().get_reels(account, **params)


def get_videos(account: str, **params: Any) -> Iterator[dict[str, Any]]:
"""A Page's long-form videos."""
return _client().get_videos(account, **params)


def get_comments(post_url: str, **params: Any) -> Iterator[dict[str, Any]]:
"""Comments on a post or reel."""
return _client().get_comments(post_url, **params)


def search_posts(query: str, **params: Any) -> Iterator[dict[str, Any]]:
"""Public posts matching a keyword."""
return _client().search_posts(query, **params)


def search_pages(query: str, **params: Any) -> Iterator[dict[str, Any]]:
"""Pages matching a keyword."""
return _client().search_pages(query, **params)
Loading