Who Embeds Your Videos? Finding Every Site That Uses Your Player From Server Logs
If you offer an embeddable player, your sign-up list is only part of the picture. Sites copy an embed code from someone else, keep using a key they shared, or embed without ever registering. The complete list is already on your server: every player load is a request in your web server's access log, and each one says which page it came from.
What a player load tells you
A typical nginx log line for a player request holds four things you need:
- The path, e.g.
/player/embed/12345/: which video. - The Referer: the page the player is embedded in. Its host name is the embedding site.
- The User-Agent: a browser, or a crawler.
- The IP address: how many different viewers there were.
Thumbnail requests are a second signal. A page that hotlinks your thumbnails lists your videos, even if it plays them somewhere else.
The method
LINE = re.compile(r'^(\S+) \S+ \S+ \[([^\]]+)\] "(?:GET|HEAD) (\S+)[^"]*" (\d{3}) \S+ "([^"]*)" "([^"]*)"')
for line in log:
ip, ts, path, status, ref, ua = LINE.match(line).groups()
if not path.startswith('/player/embed/') or status != '200':
continue
site = urlparse(ref).hostname or '(no referer)'
s = sites[site]
if CRAWLER.search(ua): s['bots'] += 1
else: s['people'] += 1; s['ips'][ip] += 1
Three details make the difference between a useful report and a misleading one:
- Exclude yourself. Your own uptime probes and test scripts load the player constantly. Ours showed up as the busiest "site" in the first report, flagged as play farming. Skip your own IPs and your monitor's user agent.
- Read every log. If the player is served under more than one domain, each has its own log file. Our first run missed a whole brand.
- Keep a week. One day is noise; seven rotated log files show which sites are steady.
Flags worth raising
| Flag | Rule we use | What it usually means |
|---|---|---|
| bot-inflated | ≥ 100 crawler loads and ≥ 50% of the site's loads | Crawlers indexing the site; its "plays" are not people |
| play-farming | ≥ 100 people loads, one IP ≥ 50% of them | One machine replaying the player |
| not-signed-up | ≥ 5 people loads, no key linked to the site | A real user you can reach out to |
| shared-key | one key on ≥ 5 unrelated sites | A key passed around or resold |
| thumbs-only | ≥ 2,000 thumbnail loads, no player loads | Your catalogue listed, videos played elsewhere |
What we found in one week
About 106,600 player loads by people against 53,900 by crawlers. One site that never signed up served our player to more than 17,000 viewers in a week. Another showed tens of thousands of "plays" in a day that were one AI crawler. Neither would have been visible from sign-ups or from the play counter alone.
Frequently asked questions
Does every browser send a Referer?
Most do for embedded iframes, though some privacy settings shorten it to the host name. That is enough for this report. Loads without one are grouped as "(no referer)".
Should I block sites that never signed up?
Usually no. They show your player to real people. Knowing who they are lets you help them, which is worth more than blocking them.
How often should the report run?
We run it every three hours over the last seven days of logs.
Related: porn API developer guide · keeping a catalogue in sync
Want a WordPress tube site that fills itself with tagged, categorized video — free?
Get your API key →