Sam Na writes practical guides on how to debug and test code with AI, review generated changes, and build dependable personal automations. RoutineOS focuses on turning AI-assisted coding into workflows readers can understand, verify, and maintain.
Debug and test code with AI more reliably by connecting four essential decisions: identifying the actual bug, defining correct behavior, reviewing the proposed change, and protecting the repair from future regressions. An AI-generated answer may look convincing, but a dependable automation needs observable evidence that its code performs the intended task.
A small automation can save hours of repetitive work until something unexpected happens. A script that organizes files might overwrite an older export. A task summary might count unfinished items as completed. A reminder workflow might send the same message twice after a retry. Sometimes the program crashes with an obvious error. Other times it finishes successfully while producing an incorrect result.
AI coding assistants make these problems easier to investigate, but they also introduce a temptation: replace the code with whatever the assistant suggests and stop when the original error disappears. That approach can leave the underlying assumption untouched. The next unusual input or minor update may reproduce the same failure in a different form.
A stronger process treats debugging, testing, code review, and maintenance as related activities. Each answers a different question. Debugging asks why the program behaved incorrectly. Testing defines what it must do. Reviewing examines the consequences of the proposed change. Regression protection checks whether future versions still satisfy the important rules.
An AI-generated fix becomes more trustworthy when its cause is understood, its behavior is tested, its side effects are reviewed, and its important requirements remain protected after later updates.
The process does not require a large engineering team. A solo creator using Python, JavaScript, ChatGPT, Claude, Gemini, Grok, GitHub Copilot, or another coding assistant can begin with a small bug report and a focused test. The same habits remain useful when a personal script grows into a scheduled workflow involving files, notifications, or connected services.
Find the real cause of an AI coding error
The first challenge in AI-assisted debugging is not generating another version of the code. It is identifying the behavior that actually failed. If the cause is misunderstood, a patch can remove the visible symptom while leaving the mistake intact.
A useful diagnosis begins with a comparison between expected and actual behavior. That comparison should be specific enough that another person, or another AI assistant, can understand which input exposed the problem and what result would count as correct.
Distinguish crashes from silent errors
A crash is relatively easy to recognize. Python may report a TypeError and show a traceback. JavaScript may throw an exception when code tries to read a property from an unexpected value. These messages identify where execution failed, although the original bad input may have entered the program earlier.
Silent errors are more deceptive. A program can run from beginning to end and still produce the wrong information. Consider a JavaScript task importer that receives the string "false" for an incomplete task. If the code uses Boolean("false"), the result is true because the string is not empty.
The script may then include unfinished tasks in a completed-task report without throwing an exception. The program's execution succeeded, but its interpretation of the data was incorrect. Asking AI to search only for runtime errors would miss the important issue.
Keep the exact message, relevant stack trace, command, and input that caused execution to stop. Check whether the named line is the source of the problem or merely the point where the program noticed it.
Save a small input and its expected and actual results. Check data types, filtering rules, ordering, and default values. A successful exit status does not establish that the result is meaningful.
Compare runtime versions, installed packages, working directories, permissions, and configuration. Rewriting a function cannot correct every missing dependency or unavailable service.
Give AI a reproducible case instead of a vague complaint
A request such as "my automation is broken" forces the model to guess which part of the process is failing. A better request includes the smallest relevant code fragment, the input that triggers the problem, the observed result, and the intended result.
For the task importer, the report might say that a record with completed set to the string "false" appears in a completed-task summary. The expected result is that the record remains excluded. This defines a testable mismatch without assuming the correct implementation in advance.
Include the source format when it matters. A field supplied as a string by a CSV import is different from a Boolean supplied by an application API. The assistant should not decide how to interpret both formats without knowing which values the real workflow is intended to accept.
Verify the cause before accepting the patch
An AI explanation is stronger when it predicts something you can check. If the diagnosis is that a string is being converted incorrectly, examining its runtime type and comparing it with the expected type should reveal a meaningful difference. A prediction gives you a way to challenge the explanation rather than accepting its confidence.
Different coding assistants may suggest different repairs for the same error. Some will focus on the immediate failing expression, while others may recommend changing surrounding code. If you are choosing between approaches from several tools, the practical comparisons in How to Fix Coding Errors with ChatGPT, Claude, Gemini, Grok & Copilot help distinguish a diagnosis supported by the evidence from a replacement that merely looks plausible. The important test is whether the suggested fix addresses the observed failure without changing unrelated behavior.
Sometimes an assistant cannot determine the cause from the available evidence. That is useful information. It tells you what to observe next, perhaps the actual value entering a function, a dependency version, or a configuration setting. Better evidence may resolve the problem more efficiently than asking several models to make increasingly confident guesses.
Start with a reproducible mismatch, not a complete rewrite request. Ask AI to explain the evidence, challenge its assumptions, and propose a narrowly scoped correction.
Generate tests that verify correct behavior
After identifying the likely cause, you need a clear definition of success. Tests provide that definition in a form the computer can check repeatedly. They are especially valuable when AI produces a patch that looks reasonable but may behave differently with empty, malformed, or unexpected input.
A good test is not a celebration of the current implementation. It is a statement about what the program is supposed to do. If the implementation changes tomorrow, the expectation should remain stable unless the actual requirement changes.
Define an independent behavior contract
A behavior contract describes accepted inputs, expected outputs, and important invalid conditions. For the completed-task importer, the contract might specify that completed is supplied as the string "true" or "false". Only "true" should include a record in the final summary.
That contract is different from one where completed is a real Boolean. Mixing the two without an approved conversion rule is a common source of mistakes. A test suite should reflect the format the application actually receives.
It also helps to decide how missing values should behave. A missing completion field might be rejected, ignored, or treated as incomplete. Those choices cannot be inferred safely from a function name. AI should surface the ambiguity rather than silently establish a new rule.
A record marked with the documented completed value is included exactly as required.
An empty task list or missing field should produce the result established by the contract, not an invented default.
Unsupported values should be handled explicitly. Where an exception is required, the test should verify that exception rather than accept any result.
Use precise assertions rather than broad success checks
An AI-generated test can pass without protecting anything important. For example, checking only that a function returns an array is weak if the actual requirement is to return the IDs of completed tasks in their original order.
Stronger assertions compare a specific result with an independently written expected value. If an input contains one incomplete task and one completed task, the test should verify that only the completed task appears. It should not merely confirm that the output exists.
Python's pytest and JavaScript's Jest support this style of testing. The syntax differs, but the principle is the same: make the assertion represent an observable requirement. For related inputs, parameterized tests can reduce repetition without losing distinct expected results.
A failing test can also have more than one explanation. An incorrect assertion, a missing package, an import path error, or a real implementation bug can all produce a red test run. Separate these possibilities before changing production code.
Make AI-generated tests challenge the implementation
A common mistake is supplying only the current code and asking AI to generate tests from it. The assistant may treat accidental behavior as correct and then produce assertions that preserve the mistake.
Instead, provide the behavior contract separately from the source. Ask for a small group of tests covering the ordinary case, the original failure, and important input boundaries. Require test names that describe the behavior they protect.
It is especially easy to mistake a high number of passing tests for confidence in the implementation. If you want to see how that happens in practice, How to Generate Unit Tests with AI for Python and JavaScript develops pytest and Jest examples that distinguish exact output checks from weak assertions, including invalid inputs and controlled dependency failures. Those distinctions help you decide whether a test would actually catch a meaningful coding mistake.
After generating the tests, inspect their assertions before running them. Ask what realistic mistake each test would catch. If the test would still pass after the wrong result was returned, the assertion needs to be reconsidered.
Tests should also avoid unnecessary contact with live systems. A function that formats an email can usually be tested without sending a message. A function that writes a report can use a temporary destination. Keeping those effects separate makes test failures easier to understand and reduces the consequences of executing generated code.
Generate tests from requirements, not from the implementation alone. Prefer a few exact assertions that can expose incorrect behavior over many superficial checks.
Review AI-generated changes before execution
A test can tell you whether a function returns an expected value under selected conditions. It may not tell you that an AI-generated patch has also changed a configuration file, introduced an external request, or replaced an existing output file. Those concerns require a review of the proposed change itself.
The most important review question is not whether the code looks clean. It is whether the code stays within the work you authorized. Even a short script can have significant consequences when it can edit files, send messages, or access connected accounts.
Inspect the full change, including removed safeguards
A diff shows additions and deletions between versions. It is useful because AI-generated edits sometimes extend beyond the function you wanted to repair. A request to change task filtering might unexpectedly modify logging, dependency configuration, or file-writing behavior.
Pay particular attention to deleted lines. An assistant may remove a validation rule, an existing file check, or a specific error message while simplifying the implementation. The revised code may look shorter while becoming less predictable.
Review newly created files as well as modified ones. A normal working-tree diff does not necessarily display the contents of untracked files. If a coding agent produced a new script or configuration file, inspect that file directly instead of relying only on a summary of changed lines.
Recognize side effects that tests may not reveal
Imagine a task exporter that writes a CSV report. The content might be correct, and a test comparing its rows might pass. Yet the function may open a fixed filename in a mode that overwrites an earlier report.
The same issue appears with scheduled notifications. The message text can be correct while the automation sends it to the wrong recipient. A local calculation test cannot independently prove that the destination and permissions are appropriate.
Before execution, identify the actions the code can perform beyond returning a value. Reading files, modifying data, contacting a service, and running shell commands require different levels of review. The most consequential operation may occupy only one line.
Check output paths, filename construction, overwrite behavior, deletion, and interrupted writes.
Identify destinations, transmitted fields, credentials, and error handling. A local task summary should not unexpectedly send private data elsewhere.
Review commands, package installation, filesystem access, and the authority granted to any coding agent.
Use AI review findings as evidence, not approval
AI code review tools can identify suspicious changes, explain possible failure scenarios, and suggest narrower corrections. That feedback becomes more useful when every finding refers to a specific location, an observable behavior, and the condition under which the problem could occur.
A warning that says "improve security" is difficult to act on. A finding that identifies a file-writing operation and explains how it may overwrite an existing export is more useful because you can compare it with the task requirements.
The risks are easier to recognize before executing generated code than after it has modified real files. For a closer examination of diff inspection, unsafe file paths, sensitive-data handling, and review tools, How to Review AI-Generated Code for Bugs Before You Run It explains which code changes deserve closer scrutiny before you grant permission to run them. The same questions apply when a generated patch passes unit tests but introduces an unapproved side effect.
Reviewers can also make mistakes. They may misunderstand the intended data format, assume missing files exist, or flag an approved behavior as suspicious. A clean AI review does not certify that code is safe. It only means the reviewer did not report a supported issue within its examined context.
Do not approve file deletion, unrestricted shell commands, exposed credentials, or unexpected network requests merely because an AI assistant reports no major concerns. The actual behavior and authorization still need to be checked.
Review the exact change, not just the AI explanation. Inspect removed safeguards, file and network operations, dependencies, and permissions before allowing generated code to affect important data.
Keep repaired bugs from returning
An automation may behave correctly today and fail again after a minor update. A change to an import function, a new data field, or a modified schedule can bring back a bug that appeared to be resolved. That is why a successful one-time test is not the end of a reliable maintenance process.
A regression test preserves an established requirement. When a future change violates that requirement, the test provides a signal before the changed behavior becomes part of the regular workflow.
Convert real incidents into lasting checks
Imagine a daily reminder automation that includes the same task twice because the import contains repeated records. After correcting the selection logic, preserve the duplicated input as a test case. The expected result should contain one candidate for that task, assuming the application's identity rules treat the repeated records as equivalent.
Without the saved case, a later refactor could remove the duplicate check without anyone noticing. With the test in place, the change can be evaluated against a requirement established by an actual failure.
An incident record can connect the observed problem, the affected code, the accepted repair, and the test that protects it. This does not require elaborate documentation. A concise record with an incident ID, a reproducible input, and the verification result is often sufficient for a small project.
Observed: Two identical imported records produced two reminder candidates in the same batch.
Expected: One candidate for the task within that batch.
Fix: Apply the approved duplicate identity rule during candidate selection.
Regression guard: Preserve the original repeated input and verify the candidate count after future changes.
Recognize problems that require persistent state
Removing duplicate records within a single execution does not guarantee that a reminder will be sent only once across separate executions. The scheduler may start another run, or the delivery service may retry after a timeout.
If the workflow must avoid repeat deliveries across runs, it needs a reliable way to recognize previously processed actions. A durable checkpoint or provider-supported idempotency mechanism may be appropriate, depending on the service and the consequences of duplication.
Even then, failure timing matters. If delivery succeeds but storing the checkpoint fails, a later retry may send another message. If the checkpoint is recorded before delivery and delivery fails, the reminder may be missed entirely.
These are different failure scenarios from duplicate imported rows. Treating them separately leads to more accurate tests and prevents a local deduplication function from being credited with guarantees it cannot provide.
Connect each accepted fix to future verification
A saved test is most useful when it runs regularly. For small projects, you may execute the suite manually before adopting a change. In a repository, automated checks can run when code is pushed or reviewed.
GitHub Actions supports workflows that install project dependencies and run tests in a controlled environment. For a Python project using pytest, the local command might simply be:
python -m pytest -qA JavaScript project using Jest can run the relevant test suite through its configured command. The important rule is to run assertions against the version of the code you intend to use, rather than relying on results from an earlier revision.
The maintenance challenge becomes clearer when a seemingly fixed bug returns after a later change. Was the original selection rule removed? Did the scheduler run twice? Or did delivery succeed while checkpoint storage failed? AI Bug-Fixing and Regression Testing for Personal Automations examines how incident records, regression cases, repeated executions, and automated checks can help distinguish these causes. That separation makes subsequent repairs more precise than simply applying another duplicate filter.
Keep ordinary tests separate from live operations. A test for reminder selection should not need access to a real email account. A test for file formatting should not replace a report in your working documents folder. Use artificial records, temporary destinations, and controlled dependencies wherever practical.
Preserve meaningful bugs as regression tests and run those checks against future revisions. Distinguish failures within one execution from retries or repeated actions across runs.
Apply a reliable end-to-end automation workflow
The individual techniques become more useful when they influence one another. The original failure tells you what evidence to collect. The expected behavior tells you which tests to write. The proposed code change determines what needs reviewing. The accepted test cases become protection for future revisions.
A practical workflow should also account for the consequences of executing an automation. Testing a pure calculation requires fewer precautions than testing a job that can send notifications, update stored records, or overwrite files.
Follow one example from incorrect output to accepted repair
Return to the task importer that receives completion values as strings. The faulty implementation converts each value with Boolean(), so the string "false" is treated as true. The program includes unfinished tasks in a completed-task summary.
The first step is to confirm the input contract. Suppose the source guarantees the literal strings "true" and "false", and only "true" means completed. Under that contract, the selection rule can compare the value directly rather than converting every non-empty string to true.
A small JavaScript function could express the intended selection like this:
function selectCompletedIds(tasks) {
return tasks
.filter((task) => task.completed === "true")
.map((task) => task.id);
}
module.exports = { selectCompletedIds };The implementation is deliberately narrow. It does not decide how to interpret missing values or unsupported formats. Those cases still require explicit rules if they may appear in the real input.
A matching Jest test can preserve the original failure condition by supplying one incomplete task and one completed task. The assertion checks the exact selected ID rather than merely verifying that an array was returned.
const { selectCompletedIds } = require("./selection.js");
test("excludes a string false completion value", () => {
const tasks = [
{ id: "T-11", completed: "false" },
{ id: "T-12", completed: "true" },
];
expect(selectCompletedIds(tasks)).toEqual([
"T-12",
]);
});
For this example, save the function in
selection.js
and the test in
selection.test.js.
The example uses CommonJS module syntax.
It does not represent an observed test run
in the reader's project.
Before execution, inspect the patch to ensure it does not also change unrelated imports, message delivery, file-writing operations, or configuration. Then run the test in a suitable project environment. If the assertion passes, add cases for other input formats the contract allows.
Finally, preserve the failing input and the approved rule as a regression guard. If a later update changes the filter back to a broad Boolean conversion, the same test can identify the mistake again.
Choose the next action based on the failure type
A reliable sequence is helpful, but not every failure needs every tool. The next step depends on what is already known and which consequence matters most.
Focus on the failing expression, the values that reached it, and the environment. Do not request a large rewrite before establishing the cause.
Define the intended output and create an assertion that would fail on the incorrect result. This prevents the current implementation from becoming the definition of correctness.
Inspect output paths, deletion, overwrite behavior, external requests, new dependencies, and requested authority. Do not grant broad access to solve a narrow problem.
Reopen the earlier incident record, run the established test, and determine whether the cause is genuinely the same as before.
Separate safe verification from real-world execution
Testing generated code requires care when the code has side effects. A test that runs a selection function on artificial records is different from a test that sends messages to real recipients. The latter may create consequences even if the test reports a failure.
For local calculations, use controlled inputs and compare returned values. For filesystem operations, use disposable directories and avoid important personal files. For network activity, replace the external dependency with a mock when the purpose is to verify local logic.
Some behavior cannot be verified completely without an authorized integration test. A mock can show that your function asked a sender to deliver a message, but it cannot prove that a real provider accepted and delivered it. Keep those verification boundaries visible in the test report.
Once local behavior has been verified, inspect the exact code revision and decide whether a restricted trial is appropriate. Important or sensitive systems may need additional review before production use.
These checks do not promise that every possible defect has been found. They provide a practical record of what was examined and which behaviors are protected. That clarity makes the next maintenance decision more manageable.
Reliability comes from connecting evidence to decisions: reproduce the failure, define the requirement, review the change, verify it in a controlled environment, and preserve the corrected behavior for later revisions.
Frequently Asked Questions
Start with a reproducible failure and an independently defined expected result. Ask AI to explain the likely cause and suggest a narrow patch, then review the proposed changes and run meaningful tests. Preserve the original failing case as a regression test when the repair is accepted.
They can propose code changes and draft tests, while some configurations also offer execution tools. Their output still needs review because a generated test may contain an incorrect expectation or miss an important condition. Actual execution results determine which assertions passed.
Use pytest for Python projects and Jest for JavaScript projects when those frameworks fit the existing environment. Both can verify expected behavior and error conditions. Choose the framework that matches your project rather than introducing unnecessary dependencies solely for AI-generated examples.
Share only information needed to investigate the problem. Remove real credentials, private records, authentication headers, and unnecessary identifying details. Use synthetic inputs when possible and follow the data-handling rules of any organization or service involved.
Identify the incorrect behavior the test is supposed to catch. Its assertion should fail when that behavior occurs and pass when the required result is produced. A controlled temporary mistake in a disposable copy can help reveal whether an important assertion is sensitive to the problem.
Unit tests verify selected behaviors under controlled conditions. They may not cover scheduler retries, unavailable services, filesystem permissions, configuration changes, or partial delivery failures. Important workflows may also need integration checks and operational monitoring.
Run relevant tests whenever code or configuration changes could affect protected behavior. Automated checks on repository updates can make this routine easier. Additional scheduled checks are useful when time-dependent behavior or changing dependencies justify them.
Conclusion: make AI coding more dependable
AI makes it easier to create scripts and solve coding problems, but speed alone does not make an automation reliable. A patch can appear convincing while changing unrelated behavior, and a passing test can provide false confidence when it checks the wrong expectation.
A dependable workflow begins with an accurate description of the failure and continues through a clear behavior contract, focused tests, careful review, and regression protection. These steps help distinguish what an AI assistant suggested from what the program has actually demonstrated.
If an automation is currently failing, begin by collecting the smallest reproducible example. If the code works but has little test coverage, define important outcomes and build a few precise assertions. If AI has proposed a significant patch, inspect its effects before authorizing execution.
For automations that already run regularly, make regression checks and incident records part of the maintenance routine. The objective is not to remove every possible risk; it is to understand what the system promises, what has been verified, and what needs attention when something changes.
Choose one Python or JavaScript automation you already use. Write down an important expected result, identify a realistic failure, and save a test that would detect it. Review any AI-proposed correction before running it, then keep the accepted test with the project.
Save this checklist for your next coding session. If someone you know is building automations with AI, share the workflow with them. Subscribe to RoutineOS for more practical ways to improve AI-assisted productivity and maintainable digital routines.
Sam Na writes RoutineOS guides about AI-powered productivity, code quality, debugging, and personal automation. His work focuses on helping readers evaluate AI-generated results, understand the behavior of their tools, and develop digital workflows that remain useful beyond the first successful run.
Topics include practical use of coding assistants, reliable software testing, automation maintenance, and decision-making around AI-generated work.
The examples and recommendations are intended to help readers understand common debugging, testing, and automation practices. Approaches described in the related articles may need to be adapted to your own code, environment, permissions, and data-handling requirements.
Before using generated code with important files, connected accounts, or consequential operations, check relevant official documentation and consider advice from a qualified developer or security professional when appropriate. Begin with safe test data and a controlled environment so you can evaluate actual behavior before relying on it in daily work.
