Updated 2026-11-17. A code security scan reads your repository, lists where an attacker could damage things, and tells you which places matter first. It runs in minutes without a consultant, so it belongs in front of a hiring decision. It will not fix anything by itself, and knowing which half it covers is the whole decision.
The useful way to judge a scanner is by its output, not its marketing. OpenAI's Codex Security, which I installed and ran on 2026-11-15, produces a report and a set of findings rather than a score. The documentation lists the pieces [1][2]:
CODEOWNERS
SECURITY.md
The exclusions row is the one a buyer should look at first. Every scan has limits: files it did not read, code paths it could not judge, checks it does not perform. A tool that shows you those limits is telling you where the next conversation has to happen. A tool that prints a clean bill of health without them is telling you nothing.
Two years ago a security scan reviewed code people wrote. Today a growing share of the code in a small company is generated, and the reviewer that mattered last year is the person who is now mostly reviewing output rather than writing it. That changes the economics in a way that shows up in two places: the volume of code arriving for review, and the fact that the agent which wrote a mistake is not a good judge of it, which is the failure mode behind agent-written code.
A scanner is not a reviewer, and it is not meant to be. It covers the class of problem that is mechanically checkable: injection, unsafe deserialization, secrets in code, dangerous defaults, known vulnerable dependencies. The judgement calls, such as whether this endpoint should exist at all, stay with a person. The point of running one before you hire is that it separates those two conversations. You find out which problems are mechanical, so the conversation about a hire starts from the real gap rather than a guess.
These are not alternatives to each other in the order they appear. The scan is the cheap piece and the hire is the expensive piece, and running the cheap piece first changes what you are hiring for. A team that scans first knows whether it needs someone to fix a backlog of mechanical issues, someone to design the next system, or nobody yet. That is a better hiring conversation than "we think we might have a problem".
The third row is where most teams end up, and it has a real cost beyond the token bill: every additional gate can block a merge at the wrong moment. Which is why the tool's exit codes matter more than its dashboard. In Codex Security, a completed scan exits 0, --fail-on-severity high returns 1 only when a high or critical finding is present, and execution or output failures return 2 [2]. That separation is what lets a pipeline fail for a security reason without failing for an infrastructure reason, and the CLI reference explicitly advises checking manifest.scan.status alongside the exit code so the two are not confused.
--fail-on-severity high
manifest.scan.status
# Fail the build only on a serious finding, and cap what the scan may spend. codex-security scan . --diff origin/main --fail-on-severity high --max-cost 5
Those two flags are the whole policy decision in one line: what stops a merge, and what stops a runaway bill. Everything else about the scan can be decided later.
Four limits are worth stating plainly, because each one is where a small team's expectations go wrong.
It will not decide what to fix. It can validate a candidate finding against source evidence, propose a patch, and verify an existing fix, which are three separate commands [1]. None of them decide priority. A high-severity finding in code nobody deploys and a medium finding on the login path need a person to say which is Tuesday's work.
It will not know your business rules. A scanner can see that a query is built from user input. It cannot see that the endpoint returns someone else's invoices because of a missing ownership check that looks correct in isolation. Rules catch patterns; intent needs a reader.
It will not cover a repository it cannot reach. If your code lives behind a private dependency manager, an internal registry, or a language the ruleset does not cover, the report will say so, and that gap is the part to fix before trusting the number of findings.
It will not run just anywhere. On the machine that produced my guide, the scan refused to start because the sandbox helper could not create the namespace it needs. That was a host configuration problem rather than a product defect, and it is the kind of thing to check before promising your team a weekly scan. The details and the fix are in the CLI guide.
This is the part that decides whether the scan was worth running. The order below is the one I would use, and it is the same order I use for a leaked key, because the two situations share a shape: something exposed, someone has to act, and the clock matters.
The mechanical steps are cheap once the decision is made. The decision is the expensive part, and it is the part no scanner sells you.
The CLI is a published package and the scans run on your machine or your CI runner, so the per-scan cost is model tokens. Codex Security documents a cost cap: --max-cost bounds spend on a model whose pricing the tool knows, and the GitHub Action example uses an estimated limit per scan while warning that a request in progress can exceed it [1][2]. Those are the tool's own settings, not prices, and the number that matters for you is the total across however many scans you run.
--max-cost
The larger cost is triage. A scan that produces forty findings and no owner produces zero fixes. The budget conversation for a small team is therefore not "what does a scan cost" but "who reads the output", and that is the question what an AI engineer costs is really answering. If the answer is nobody, the scan is a compliance artefact rather than a security control, in the same way an agent doing work nobody asked for is activity rather than progress.
A security scanner covers the classes of bug that can be listed. The rest of review, meaning style, duplication, dead code, null handling and the local conventions of a codebase, is what a code review tool covers, and those tools have also been shipped open source in the last week. A practical pairing for a small team is one security scan on a schedule, one review pass on every pull request, and a person who reads both. I wrote up the review half separately: Open Code Review, the CLI that resolves rules per language.
If your team is already running either of these, the question to ask this week is not which tool is better. It is which finding from the last scan is still open and has no name against it.
Is a security scan the same as a penetration test? No. A scan reads your code and configuration for known classes of weakness. A penetration test has a person attempt an attack against a running system. A scan is the cheap, repeatable first step and it does not replace the other.
Does it replace code review? No, and the two cover different things. Review catches intent, design and convention problems that a rule cannot express. The review tool I wrote up is a complement to a scanner, not a substitute.
Who fixes the findings? A person on your team, by name. Some tools can propose and verify a patch, which shortens the work, but the ownership decision is not automatable in any way that survives an audit.
Can it run on every pull request? Yes, and that is where it pays for itself. Scoping to the change under review rather than the whole history is the difference between a two-minute gate and a twenty-minute one.
What does my team need to start? A repository, a machine or CI runner that can run the CLI, and one person who will read the report. That is the entire prerequisite list, and the second item is the one that surprises people because of host sandbox settings.
What if the report says the scan covered nothing? Then it is telling you the truth and the gap is configuration: a path that was excluded, a language without rules, or a checkout the tool could not reach. Exclusions are listed in the report for exactly this reason.
[1] OpenAI, Codex Security README, fetched 2026-11-17. Source of what a scan produces, the findings and owner suggestions, the SECURITY.md drafting and threat-model export, the validate, patch and verify-fix commands, the tracker connectors, and the GitHub Action setup including the cost limit and its warning.
[2] OpenAI, Codex Security CLI reference, fetched 2026-11-17. Source of the exit-code table used in the CI row, the manifest.scan.status warning, --max-cost, and the scan scope behaviour for diff and working-tree runs.
[3] OpenAI, Codex Security CI guide, fetched 2026-11-17, referenced for running scans in continuous integration.
[4] OpenAI, Codex Security GitHub Action guide, fetched 2026-11-17, referenced for severity thresholds, pull request checks and runner requirements.
[5] Alibaba, Open Code Review README, fetched 2026-11-17, referenced for the review half of the pairing described above and for the rule scope it claims.
If you want a scan, a review pass and an owner for the findings set up inside your own repository, send a written intake and we will tell you which of the two to run first for your stack.
Maxpertise is an AI-native engineering company. We embed native AI engineers inside your team, live in about 10 days. Please enable JavaScript to view the site, or email contact@maxpertise.net.