How we build and validate this data

Everything here comes from public web pages. We read the public integration directories and marketplaces that software companies publish: app stores, the catalogs of automation platforms, and the “Integrations” page on a product’s own site. Most list the apps that connect to that product; some list partners or catalog entries instead, and we record those as presence on the directory, not as a connection.

Our crawler identifies itself as LeftHook-CatalogIngest with a link to lefthook.com. It limits how fast it asks any one site. It does not log in, and it does not get around a bot challenge: a directory behind a login or a challenge is recorded as unreadable and left out. If you run a directory and want us to stop reading it, tell us and we will.

We do not buy lists, and we do not use private data. Company details that go beyond a directory’s own page (size, location) exist for only a small number of vendors.

We aim to re-read every directory about once a week. Each read compares the directory with what we had before, so we record what was added, renamed and removed, and when.

The explorer does not update the moment that happens. It shows a published snapshot, and the snapshot id and date appear on the explorer page. The data is recent, not live.

A directory lists apps by whatever name it likes. We keep that raw listing exactly as we read it, then match it to a product in a separate step. A listing is not a product, and the same product can appear under different names on different directories.

• The name suggests, the company website decides. Two companies can share a name. When a name could mean more than one company, we leave the listing unmatched rather than guess.
• Fuzzy matches are never linked on their own. They go to a review queue.
• New products need evidence. We create a product automatically only when the same name appears on several independent directories. Plugin and extension stores (WordPress, VS Code, Shopify and similar) do not count toward that, because short names repeat there by coincidence. Generic words such as “CSV” or “PDF” never become products.
• Duplicates are folded, not deleted. When two products turn out to be one, we merge them, and the merge can be undone.

Nothing a scrape produces is shown until it passes these checks.

1. Does it look like a directory of software? Before we start reading a directory regularly, the names we extracted are checked for page furniture (“Privacy Policy”, “Careers”), junk, descriptions stuck onto names, and companies that are not software. Rules run first, then a model reads a sample.
2. Did we read all of it? When a directory states how many apps it has, a read that falls short of that number is treated as incomplete. We keep what it added and remove nothing.
3. Did something change too much? If a read would remove more than a small share of a directory, or comes back empty, or far smaller than before, we do not apply it. It becomes a hold. Unusually large growth is held too.
4. A person reviews every hold. Each one comes with a check of whether the pages still load. If the directory stopped listing an app but its page still works, we mark it “unlisted” rather than removed.

Removed and unlisted listings are kept as history. They do not count as listed.

Counts on the explorer are counts of what we read and matched. They are not counts of everything that exists. The words we use mean specific things:

Listed

We saw the product on that directory’s public page the last time we read it successfully.

Not found in our index

We did not find it. That is all it says. It does not mean the directory does not list it.

Unknown

We read a listing but could not tell which product it is, or we do not read that directory. Unknown is never shown as “not listed”.

Provisional

A product our pipeline created rather than one we curated by hand. Most come from the same name appearing on several independent directories; a few come from a reviewed list or an audit. It is real data, checked less closely than a curated product.

A listing count is not a count of integrations. One integration can sit behind several listings, and a listing is only what a directory shows. We also do not read a product’s disappearance from a directory as the product shutting down.

• We only cover directories we found, can read, and may read. Some are missing for that reason.
• A large share of listings are not matched to a product yet. A missing link in the explorer is often missing knowledge, not a missing integration. In spot checks, partners marked “not found in our index” turned out to list the product on their own site.
• Categories, websites and company details are partial. Many products have no category yet.
• Products created by our pipeline can still be wrong: two products merged by mistake, or one split in two. We find these through review and corrections.
• The explorer is a prototype and says so.

If a product is wrong, missing or merged with the wrong company, or you run a directory or a product and want an entry changed or removed, use the contact form. Send the product name and the page or listing you mean. We will look at it and reply.