← All guides AI Release
RU Subscribe to @ai_release1

How to Run an OSINT Self-Audit: Check What Personal Data Is Publicly Available Using Watson and Free Tools

Updated: 28.10.2026 · AI Release · @ai_release1
How to Run an OSINT Self-Audit: Check What Personal Data Is Publicly Available Using Watson and Free Tools

TL;DR

Requirements and setup

Illustration: Requirements and setup
Illustration: Requirements and setup

Install the toolchain:

# Debian/Ubuntu/WSL
sudo apt update && sudo apt install -y exiftool jq curl pipx
pipx ensurepath

# macOS
brew install exiftool jq pipx

Example: on Ubuntu 22.04, exiftool -ver prints a version string such as 12.40, and jq --version prints jq-1.6.

What to do: verify all four tools respond.

Why: a missing binary fails silently later, when you are already 20 minutes into the audit.

How to check:

python3 --version && exiftool -ver && jq --version && pipx --version

Expected: four lines of version output, no command not found. Exact numbers differ by distribution — that is fine.

Create the working folders and the breach log:

mkdir -p ~/osint-audit/{raw,report} && cd ~/osint-audit
printf 'email,breach,date,data_classes\n' > raw/breaches.csv

How to check: ls raw prints breaches.csv.

Step 1. Map what is already indexed about you

What to do: build a fixed list of search queries (called "dorks" — search operators such as site: and filetype: that narrow results) and open each one in a browser. Store the queries in a file so the audit is repeatable in 30 days.

cd ~/osint-audit
cat > dorks.txt <<'EOF'
"<your-full-name>"
"<your-full-name>" site:linkedin.com
"<your-full-name>" filetype:pdf
"<your-email>"
"<your-email>" filetype:xlsx
site:pastebin.com "<your-email>"
EOF

Replace <your-full-name> with your name exactly as it appears on your passport or contract, and <your-email> with the address you are auditing.

Convert the queries into clickable, URL-encoded links:

while read -r q; do
  printf 'https://duckduckgo.com/?q=%s\n' "$(printf '%s' "$q" | jq -sRr @uri)"
done < dorks.txt > raw/dork-urls.txt

Why: jq -sRr @uri percent-encodes quotes and spaces, so the query survives the trip through the browser address bar. Hand-typed dorks break on the first ".

How to check: wc -l raw/dork-urls.txt prints 6, and the first line of head -1 raw/dork-urls.txt looks like https://duckduckgo.com/?q=%22John%20Doe%22.

Open each URL, and for every hit that is genuinely you, append one row to the findings log:

cat >> raw/breaches.csv <<'EOF'
<your-email>,<breach-name>,<breach-date>,<data-classes>
EOF

Example: <your-email>,ExampleForum2019,2019-04-02,email addresses; hashed passwords.

Next, check known breaches: open https://haveibeenpwned.com, enter the same address, and copy each listed breach into raw/breaches.csv using the exact fields the site shows. The site's API requires a separate key — check the documentation if you want to automate this step.

Why this works: breach lists are public and indexed, so they surface your old addresses faster than any other source.

How to check: column -s, -t raw/breaches.csv shows a readable table with one header row plus one row per breach.

Step 2. Enumerate usernames and strip metadata from your own files

Illustration: Step 2. Enumerate usernames and strip metadata from your own files
Illustration: Step 2. Enumerate usernames and strip metadata from your own files

What to do: find which sites still hold an account under your handle, then read the metadata out of the files you publish.

pipx install sherlock-project
sherlock <your-username> --timeout 10 --print-found | tee raw/sherlock.txt

--timeout 10 is the per-site request limit in seconds; --print-found hides the "[*] checking…" noise and keeps only positive matches.

Example: a run that finds 14 accounts prints 14 lines starting with [+], e.g. [+] GitHub: https://github.com/<your-username>.

Why: an old handle tied to a forum you forgot about is a direct path from your name to your posts, and often to a reusable password.

How to check: grep -c '^\[+\]' raw/sherlock.txt returns the number of live accounts. Anything you no longer use goes on the deletion list.

Now scan the files you actually publish — CVs, photos, contracts — for hidden author and GPS fields:

exiftool -r -csv -Author -Creator -Artist -GPSLatitude -GPSLongitude -CreateDate \
  ~/Downloads > raw/metadata.csv

Why: word processors and phones write your real name and shooting location into the file, and those fields survive re-upload to job boards.

How to check: awk -F, 'NR>1 && ($2!="" || $3!="" || $4!="")' raw/metadata.csv lists only files that still carry an author name. An empty result means the files are clean.

Example: a hit looks like photo.jpg,John Doe,,,55.7558,37.6173 — author plus coordinates. Remove them before the next upload:

exiftool -all= -overwrite_original ~/Downloads/photo.jpg

How to check: re-run the awk command; the file disappears from the output.

If you later automate these loops with an agent that has access to your mailbox or cloud drive, read AI agent security: protecting your data during automation first — an audit bot with write access can leak exactly what you are trying to contain.

Step 3. Consolidate the findings in Watson and verify the result

Illustration: Step 3. Consolidate the findings in Watson and verify the result
Illustration: Step 3. Consolidate the findings in Watson and verify the result

What to do: merge the two raw sources into one normalised CSV, import it into Watson, and tag each row with a severity.

Create normalize.py in ~/osint-audit:

import csv, pathlib

fields = ["finding", "type", "source", "url_or_handle", "exposure", "action"]
rows = []

for r in csv.DictReader(open("raw/breaches.csv")):
    if not r.get("email"):
        continue
    rows.append({"finding": r["breach"], "type": "breach", "source": "HIBP",
                 "url_or_handle": r["email"], "exposure": r["data_classes"],
                 "action": "rotate password + enable 2FA"})

for line in open("raw/sherlock.txt"):
    line = line.strip()
    if not line.startswith("[+]"):
        continue
    rows.append({"finding": "account exists", "type": "username",
                 "source": "sherlock", "url_or_handle": line.split()[-1],
                 "exposure": "public profile", "action": "review or delete"})

out = pathlib.Path("report/findings.csv")
with out.open("w", newline="") as f:
    w = csv.DictWriter(f, fieldnames=fields)
    w.writeheader()
    w.writerows(rows)
print(f"{len(rows)} findings -> {out}")
python3 normalize.py

Example: with 3 breaches and 14 live accounts, the script prints 17 findings -> report/findings.csv.

Import report/findings.csv into your Watson workspace, then de-duplicate on url_or_handle and set severity: breach with password data = high; username on a site you still use = medium; abandoned account = low. The import button, column-mapping screen and export format differ per build — check the documentation for the exact path, then write it into report/watson-notes.txt.

Why: Watson gives you one deduplicated view instead of three files, so the deletion list is generated once and re-checked later.

How to check: in Watson, the row count equals the number the script printed, and the severity column has no empty cells. Then export the report and confirm the export reopens:

head -3 report/findings.csv

Expected: a header line plus two data lines containing a real handle or email, not ,,,.

Finally, close the loop on the two highest-severity items and log the date you did it. Re-run python3 normalize.py after 14 days — the same script on the same inputs must return the same or a smaller count.

Before you store the report on a laptop, note that it now contains your addresses, handles and breach history: encrypt the volume, and if you use Windows, keep the recovery key escrowed — see BitLocker: how to recover your key and avoid losing data.

Audit checklist:

Troubleshooting

Problem. pip install sherlock-project refuses to install.

pipx install sherlock-project
pipx ensurepath
exec $SHELL -l
sherlock --version

Problem. ExifTool reports nothing for a folder full of photos.

ls -1 ~/Downloads | head
exiftool -r -csv -Author -Artist ~/Downloads | head -5

Problem. The breach API returns an error instead of JSON.

Summary and next steps

You now have a reproducible four-file pipeline (dorks.txt, breaches.csv, sherlock.txt, findings.csv) plus a severity-tagged Watson report, and every high-severity item has a dated remediation. Re-run python3 normalize.py in 14 days and the row count should fall or stay flat — if it grows, a new account or breach has appeared. If you plan to widen the audit to exported message archives and need to clean them before analysis, use How to prepare data for a neural network: dataset cleaning and labeling.

FAQ

Is it legal to run this audit on myself?

Yes — every step above reads public sources or data tied to your own accounts. Do not point sherlock or the dorks at other people's handles without their consent.

How long does data take to disappear from search results?

It varies by engine and by page; there is no fixed number. Removing the underlying page or account is what actually makes the result drop out, so log the removal date in findings.csv and re-check after 14 days.

Does deleting an account remove the data a breach exposed?

No. A breach copy exists outside the service's control, so the only real fix is rotating the password everywhere it was reused and enabling 2FA.

Do I have to pay for any of these tools?

The commands in this guide use free packages (exiftool, jq, pipx, sherlock-project). Watson's plan limits and the HIBP API key are separate — check the documentation for both before you scale the audit up.

🤖 AI summary

Related guides

AI agent security: protecting your data during automation

Разбираем риски использования ИИ-агентов и даём практические советы по безопасной автоматизации задач.

Open-weight AI models: how to run them for free in 2026

Разбираем, чем open-source модели отличаются от закрытых, и даём список актуальных LLM с открытыми весами для локального

How to prepare data for a neural network: dataset cleaning and labeling

Как подготовить данные для обучения нейросети: очистка, нормализация и разметка датасета с примерами.