AI Bug-Fixing and Regression Testing for Personal Automations

Imagine a personal automation that sends reminders for tasks due today. You built it with help from an AI assistant, ran it a few times, and added it to your daily schedule. One morning, the script sends the same reminder twice. You ask the assistant to fix the duplication, apply the suggested change, and see a single reminder on the next run.

A week later, the duplicates return after a small update to the task-import function. The original fix worked for one example, but nothing in your maintenance process required later changes to preserve that behavior. The problem was resolved temporarily without becoming a permanent condition that the project could check.

This is why regression testing matters. A regression is when something that previously worked stops working correctly after a change. A regression test preserves an expected behavior so future changes can be checked against it. When the test comes from a real incident, it also documents a mistake your automation has already made.

AI can make this workflow easier by summarizing error records, turning sanitized incident details into test cases, comparing a patch with the previous version, and helping interpret later failures. It cannot decide which business rule is correct or prove that a test ran unless you supply an actual result.

We will follow one practical scenario involving duplicate task reminders, using Python to prevent duplicate entries within an import and JavaScript to illustrate a separate problem that arises across repeated executions. Then we will connect those checks to a lightweight maintenance routine and an optional automated test workflow.

Build a repeatable bug-to-regression workflow

A personal automation can break without anyone making a deliberate change to its central function. An imported file may contain a new format, a scheduled job may run twice, or a connected service may retry an operation after a timeout. Sometimes the code is wrong; other times the code is relying on an assumption that no longer holds.

The maintenance workflow needs to handle both cases. It begins with the incident rather than a new AI prompt, and it ends only after the corrected behavior has been made observable and repeatable.

Understand the difference between a fix and a regression guard

A fix is a change intended to remove a particular failure. A regression guard is a lasting check that the important behavior remains correct. They belong together, but they do different jobs.

Suppose the reminder script receives duplicate rows from an exported task list. The fix might identify repeated task IDs and avoid creating duplicate reminder candidates. The regression test then feeds the same duplicated rows into the selection function and checks that only one candidate is returned.

If another developer or AI agent later replaces that selection function, the test can detect the reappearance of duplicate candidates. It does not depend on remembering an old chat conversation or recalling why a particular conditional statement was added.

Give every incident a place in one maintenance loop

A useful workflow does not require expensive software or a complicated issue tracker. A small project folder, a text-based incident record, and a few repeatable commands can be enough. The important choice is to keep evidence together instead of scattering it across terminals, AI chats, and temporary files.

Assign an incident identifier, such as BUG-014, whenever a failure deserves investigation. That identifier can appear in the incident note, the test name, and the change description. It lets you connect a failed automation run with the later decision that resolved it.

Keep the record short enough to maintain. Note the expected behavior, observed behavior, affected operation, smallest relevant input, accepted patch, and verification outcome. Add environment details when they matter, but do not turn every minor issue into an extensive report.

Choose the right level of maintenance for the automation

Not every personal script deserves the same amount of testing. A one-time calculator that reads no private files has different consequences from a scheduled process that updates a calendar or sends messages. Choose the amount of regression protection based on the effect of an incorrect result.

For a small deterministic function, one precise test may preserve the important behavior. For an automation that talks to external services, you may need separate tests for selection logic, delivery behavior, failure handling, and stored state. Those checks should use safe data and controlled dependencies whenever possible.

The goal is not to test every implementation detail. It is to preserve the commitments the automation makes: which records it processes, which records it skips, and which actions it is allowed to perform.

Record the failure and preserve a reproducible case

The quality of your bug history depends on what you preserve before changing the code. Once a patch has been applied, it can be surprisingly difficult to recreate the original conditions. Logs may be replaced, temporary inputs may disappear, and the original AI conversation may contain several different proposed explanations.

Capture the smallest useful evidence while the failure is still understandable. You do not need to keep every line of a large log. You need the information that distinguishes this incident from other failures that merely look similar.

Write a bug report that describes an observable mismatch

Imagine that your daily reminder automation is expected to create one candidate reminder for each distinct open task due on the selected day. A task export contains the same task twice, perhaps because two imports were combined. Your existing selection function includes both entries in the output.

The report should identify that exact mismatch. Avoid descriptions such as "the AI automation is unreliable" or "notifications are broken." Those statements do not identify the record involved, the expected count, or the boundary at which duplication occurred.

You should also distinguish the candidate-selection problem from actual message delivery. Two candidate records may cause two sends, but they are not proof that every duplicate notification originates in the candidate selector. The scheduler or the delivery service could be involved too.

Save the input, runtime, and previous version

The test input should be small enough to share with an AI assistant safely. Replace real task names, personal information, and account identifiers with artificial values while preserving the structure that caused the bug.

In this case, the task ID matters because duplicate detection uses that field. The actual identity of the user's task does not matter. A fabricated ID such as t-17 and a title such as "Pay invoice" are enough to demonstrate the behavior.

Record the script version or Git commit that produced the failure. When relevant, keep the Python or Node.js version and the configuration controlling the automation's schedule. That information becomes useful when a bug appears only after a runtime or dependency change.

Do not replace the original failing example with an AI-generated "cleaner" version before confirming that the cleaned example still produces the bug. A reproduction that no longer fails cannot establish the original cause.

Ask AI to turn the evidence into an incident summary

AI can help organize a messy failure report, especially when you have terminal output, a scheduled-job log, and a small code fragment. Give it the observations and ask it to separate confirmed facts from possible explanations.

That distinction matters because AI may confidently blame a scheduler for duplicate sends when the only evidence available is a duplicated data row. Your incident record should not convert a hypothesis into a confirmed root cause.

Turn a duplicate-reminder bug into a Python regression test

Now we can turn the incident into a lasting check. Our example assumes a daily task export contains records with an ID, title, due date, and status. The selector should return open tasks due on the requested day, in their original order.

The new requirement is that repeated rows with the same ID and the same title represent one reminder candidate. If two matching rows use the same ID but disagree on the title, the function should report the conflict rather than silently choosing one.

Reproduce the original defect in the selector

The simplified original function filters records by due date and status. Its logic works for ordinary rows but has no rule for repeated task IDs.

def due_reminders(rows, day):
    return [
        {
            "id": row["id"],
            "title": row["title"],
        }
        for row in rows
        if row["due"] == day
        and row["status"] == "open"
    ]

If an imported list contains two identical qualifying records, the list comprehension returns two candidates. The code is not experiencing a Python exception. It is doing exactly what the expression requests, which differs from the newly clarified requirement.

Before modifying this function, write a test that captures the desired behavior. On the original implementation, that test should fail because it receives two candidates instead of one. This is a meaningful red state: it demonstrates that the regression test can recognize the bug.

Apply a narrow fix with explicit duplicate rules

The repaired function below keeps a dictionary of task IDs already selected. It ignores identical repeated records while rejecting conflicting titles for the same ID. Records that are not open or are due on another day remain outside the result.

Save the implementation in a file named reminders.py. This example assumes that each row is a dictionary and that the supplied day uses the same date-string format as the imported records. Those input assumptions are part of this small example, not universal production validation rules.

def due_reminders(rows, day):
    seen = {}
    result = []

    for row in rows:
        if (
            row.get("due") != day
            or row.get("status") != "open"
        ):
            continue

        task_id = row.get("id")
        title = row.get("title")

        if (
            not isinstance(task_id, str)
            or not task_id.strip()
        ):
            raise ValueError("Active task needs an ID")

        if (
            not isinstance(title, str)
            or not title.strip()
        ):
            raise ValueError("Active task needs a title")

        title = title.strip()

        if task_id in seen:
            if seen[task_id] != title:
                raise ValueError(
                    "Conflicting duplicate task ID"
                )

            continue

        seen[task_id] = title

        result.append({
            "id": task_id,
            "title": title,
        })

    return result

This patch makes the duplicate rule visible. It does not install a package, change the scheduler, or introduce a delivery service. It also avoids using a task title as the unique key, because two different tasks might legitimately have the same title.

One important design choice remains: the function uses the task ID as an identity within the imported batch. If the source system recycles IDs or uses a different identity rule, that requirement must be revisited.

Notice that this function creates candidates; it does not send reminders. That separation keeps its behavior deterministic and makes the duplicate-row problem easier to test. Delivery across separate runs requires a different kind of protection, which we will address next.

Preserve the real bug as a permanent pytest test

The test suite below uses the incident's duplicated record as its first case. It also checks that unrelated tasks are excluded, conflicting IDs are rejected, and a selected task cannot have a missing ID. Each test represents a separate behavior that matters to the automation.

Save the test code as test_reminders.py in the same example directory as reminders.py.

import pytest
from reminders import due_reminders


DAY = "2026-10-26"


def test_duplicate_rows_make_one_candidate():
    rows = [
        {
            "id": "t-17",
            "title": "Pay invoice",
            "due": DAY,
            "status": "open",
        },
        {
            "id": "t-17",
            "title": "Pay invoice",
            "due": DAY,
            "status": "open",
        },
    ]

    assert due_reminders(rows, DAY) == [
        {
            "id": "t-17",
            "title": "Pay invoice",
        }
    ]


def test_ignores_closed_and_other_day():
    rows = [
        {
            "id": "a",
            "title": "Archive",
            "due": DAY,
            "status": "done",
        },
        {
            "id": "b",
            "title": "Plan",
            "due": "2026-10-27",
            "status": "open",
        },
    ]

    assert due_reminders(rows, DAY) == []


def test_conflicting_duplicate_is_rejected():
    rows = [
        {
            "id": "t-17",
            "title": "Pay invoice",
            "due": DAY,
            "status": "open",
        },
        {
            "id": "t-17",
            "title": "Cancel invoice",
            "due": DAY,
            "status": "open",
        },
    ]

    with pytest.raises(
        ValueError,
        match="Conflicting duplicate",
    ):
        due_reminders(rows, DAY)


def test_active_missing_id_is_rejected():
    rows = [
        {
            "title": "Pay invoice",
            "due": DAY,
            "status": "open",
        }
    ]

    with pytest.raises(
        ValueError,
        match="needs an ID",
    ):
        due_reminders(rows, DAY)

From the example project directory, run the tests using the Python environment where pytest is installed.

python -m pytest -q test_reminders.py

The four displayed regression cases were executed with pytest against the corrected function, and all four passed in that local check. That result verifies the particular example shown here. It does not establish that a different task importer or a real reminder service will behave correctly.

For workflows involving files, place test exports in temporary directories instead of pointing at your real documents folder. pytest provides a tmp_path fixture for separate temporary test directories. It is useful when you want to preserve a regression involving a file import without rewriting production data.

Protect repeated JavaScript automation runs

Duplicate rows within one import are only part of the problem. A scheduled automation may run again after a restart, a service may retry a request, or someone may manually start the same task twice. Even with a perfectly deduplicated candidate list, the next execution can select the same task again.

The difference matters because preventing duplicate entries inside one batch is not the same as preventing duplicate actions across multiple runs. A function's in-memory set disappears when the process exits. A later execution cannot remember what happened earlier unless durable state is maintained.

Distinguish batch deduplication from repeat-run protection

Suppose the reminder selector produces one candidate for task t-17. The sender delivers it, then the scheduler starts the job again. The candidate selector still returns t-17 because the task remains open and due today.

From the selector's perspective, this is correct. Its job is to choose due tasks from the input. Preventing another delivery requires knowing whether the reminder was already sent for the relevant reminder period.

This usually means recording a durable delivery key after an operation has been confirmed. A key could combine the task ID with the reminder date or another defined notification window. Using the task ID alone may accidentally suppress a legitimate reminder on a later date.

Test the decision to skip an already recorded reminder

The following JavaScript function shows the selection part of this process. It receives reminder candidates and a collection of IDs previously recorded as sent. It returns candidates whose IDs are not in that collection.

Save the function as checkpoint.cjs. The use of CommonJS here keeps the small example compatible with a straightforward Node.js setup.

function chooseUnsent(reminders, sentIds) {
  const sent = new Set(sentIds);

  return reminders.filter((item) => {
    return !sent.has(item.id);
  });
}

module.exports = { chooseUnsent };

This function assumes the caller supplied a list of already deduplicated candidates, valid IDs, and the correct delivery history for the relevant period. It does not read or save a checkpoint. Those responsibilities belong to other parts of the automation.

The matching test checks that an ID previously recorded as sent is excluded, while an unrecorded candidate remains available. It also checks that an empty history does not suppress every reminder.

Save the following Jest tests as checkpoint.test.js. The test file imports the CommonJS module from checkpoint.cjs.

const { chooseUnsent } = require("./checkpoint.cjs");

describe("reminder checkpoint filtering", () => {

  const reminders = [
    {
      id: "t-17",
      title: "Pay invoice",
    },
    {
      id: "t-19",
      title: "Review draft",
    },
  ];

  test("skips an already recorded reminder", () => {
    const result = chooseUnsent(
      reminders,
      ["t-17"]
    );

    expect(result).toEqual([
      {
        id: "t-19",
        title: "Review draft",
      },
    ]);
  });

  test("keeps all reminders with no history", () => {
    expect(
      chooseUnsent(reminders, [])
    ).toEqual(reminders);
  });

});

The JavaScript arrow functions in the HTML source use the correct => character sequence for display in a code block. When Blogger renders the HTML, readers will see the ordinary JavaScript arrow operator.

This example is intended for a project with Jest installed. The core filtering behavior was checked using Node.js assertions, and the JavaScript syntax was checked separately. The complete Jest suite was not executed in an installed Jest environment, so you should run it in your configured project.

Jest's asynchronous testing documentation is also useful when the surrounding automation sends messages or writes state asynchronously. Those tests must wait for the relevant promise before judging the outcome.

Handle partial failures and delivery acknowledgments

A durable checkpoint introduces a harder question: when should the automation record that a reminder was sent? If it records success before the message is delivered, a later failure may cause the reminder to be skipped permanently. If it records success after delivery, the process could crash between delivery and checkpoint storage, leading to another send on retry.

This is a partial-failure problem. The automation has two separate operations whose outcomes may not be committed together: delivery and local state persistence. A simple file of sent IDs cannot guarantee exactly-once delivery under every failure condition.

For low-risk personal reminders, you may accept occasional duplicates while prioritizing delivery. For an operation with financial consequences, account changes, or other sensitive effects, you may need provider-supported idempotency keys, durable queues, transactional safeguards, or independent engineering review.

An idempotent operation is designed so repeating an equivalent request does not create an unintended additional effect. Where an external service supports an idempotency mechanism, use the documented contract rather than assuming a local filter is enough.

Ask AI to review the failure sequence, not merely the final function. A useful test plan considers a successful send, a rejected send, a failed checkpoint write, and two overlapping executions. Each scenario may need a different observation to establish that the workflow behaves as intended.

Compare before-and-after behavior and approve a fix

Once a patch is proposed, the next question is whether it repairs the incident without changing something else that users depend on. An AI assistant may produce a shorter implementation or a more elegant approach, but those qualities do not establish that existing behavior has been preserved.

Before approving the change, compare the failing case, a representative normal case, and any surrounding behavior that could be affected by the patch. Keep these observations connected to the same code revision so later maintenance does not mix results from different implementations.

Compare the same inputs against both versions

A useful before-and-after comparison keeps the environment and input as consistent as possible. For BUG-014, feed the identical duplicated task rows into the previous selector and the repaired selector.

The previous version returns two candidates. The repaired version should return one. That establishes the intended change in behavior. Next, pass a list with two distinct eligible tasks and confirm that both still appear.

Finally, check the rule for conflicting duplicate IDs. If the same ID appears with different titles, the current contract requires an explicit error rather than silently choosing the first entry. That test protects a different condition from the original duplicate-row defect.

Inspect the changed code without accepting unrelated edits

The accepted fix should have an understandable scope. In the Python example, the change adds task-ID tracking and explicit handling for repeated records. It does not need to rewrite the scheduling mechanism, change message templates, or install another dependency.

If an AI assistant proposes a broader refactor, ask which changes are required for the incident and which are optional improvements. Separating those decisions makes a later regression easier to investigate.

When using Git, a diff helps identify additions and deletions. Keep a version of the code from before the repair, and record the accepted change. An earlier good version becomes a practical recovery point if a new change causes unexpected effects.

The regression test should be reviewed independently of the implementation. If the AI changes both the code and the expected test output merely to produce a passing result, the test may no longer represent the intended behavior.

Interpret a failed regression test before updating it

A failed test is a signal, not a diagnosis. The implementation may be wrong, the environment may be different, the test may contain an incorrect assumption, or a product requirement may have changed deliberately.

If a test starts failing after a legitimate requirement change, update the behavior contract and document that decision. Then revise the test to reflect the new requirement. Do not delete the test simply because it blocks the latest AI-generated patch.

In contrast, if the requirement has not changed, a failure demands investigation. A passing test from last week is not enough to approve this week's code. Run the test against the exact version you intend to use.

Run regression checks automatically

The value of a regression test increases when you do not have to remember to run it every time. Automated checks can run when code changes are pushed, before changes are merged, or at selected maintenance intervals.

The important distinction is between testing an automation and running the automation's real actions. A regression workflow should generally operate on synthetic input and mocked dependencies. It should not send real reminders or update production accounts simply to verify the selection logic.

Start with one local command that checks the project

Before adding any scheduled checks, make sure the relevant tests can be run locally with a documented command. For a Python-only project, this may be as simple as invoking pytest from the project directory.

python -m pytest -q

For a JavaScript project, use the test command provided by its package configuration. If Jest is installed and configured, you can run the intended test files from the command line.

npx jest checkpoint.test.js --runInBand

If both languages are used, maintain separate dependency environments and record the commands for each. It is better to have two reliable, understandable commands than one complex script that hides which test suite failed.

When a test fails, save the command, code version, and relevant error output. Those details help distinguish a genuine regression from a missing dependency or configuration problem.

Add a GitHub Actions check without production credentials

For code stored in a GitHub repository, GitHub Actions can run a regression suite whenever relevant code is pushed or reviewed. The official GitHub guide to building and testing Python explains how to set up a Python environment and run tests in a workflow.

The example below assumes a small Python project whose tests require only pytest. Save the configuration as .github/workflows/regression.yml. It installs pytest in the workflow environment and runs the tests without any delivery credentials.

name: Regression checks

on:
  push:
  pull_request:
  workflow_dispatch:

permissions:
  contents: read

jobs:
  python-tests:
    runs-on: ubuntu-latest

    steps:
      - name: Check out code
        uses: actions/checkout@v6

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install test framework
        run: python -m pip install pytest

      - name: Run regression tests
        run: python -m pytest -q

This is a minimal example, not a complete workflow for every repository. If the tests require project dependencies, install them from the project's approved dependency definition as well. Review third-party actions and repository permissions according to your security requirements.

This workflow does not run your real reminder sender. It runs local tests designed to use artificial records. There should be no reason to add a live email password or messaging-service token merely to test reminder selection.

GitHub Actions also supports scheduled workflows, but the schedule belongs to a different decision. If your test suite already runs on each code change, adding a daily schedule may provide little value unless external dependencies, runtime updates, or time-dependent behavior can change between commits.

Use test results as maintenance signals

An automated test result should help you decide what to do next. A failed run needs investigation, but the report must first identify whether the failure came from the test assertion, an import, a missing dependency, or an unavailable service.

Avoid turning every warning into an immediate production fix. A transient problem in a test runner may require a rerun or environment investigation. Changing application code without understanding the failure can introduce a new regression.

If the automated check detects a previously repaired bug, reopen the relevant incident record and connect it to the new code change. That preserves the relationship between a historical failure and its recurrence.

Maintain a simple AI-assisted bug history

A regression suite protects behavior, while an incident history explains why the protection exists. Both become more useful as a project changes. Without that context, someone may remove a seemingly redundant test and accidentally recreate an old failure.

You do not need to preserve the entire AI conversation for every bug. Store the decision-making facts: what happened, what was required, what changed, and what evidence showed the fix was accepted.

Connect incident IDs, code changes, and test names

Use a simple convention so you can find all material related to one issue. A filename such as BUG-014.md might describe the duplicate-reminder case. Its resolution can mention the accepted Git commit and the regression test that protects the behavior.

When the same symptom appears months later, compare the new incident with the earlier record. It could be the same bug returning, or it could be a new defect in a different part of the workflow.

For example, one case may concern duplicate input rows, while another concerns a scheduler restarting after partial delivery. Both can cause repeated notifications, but they require different regression checks.

Use AI to summarize change history without rewriting the facts

AI can help turn a collection of incident notes, diff summaries, and test results into a concise maintenance update. That is useful when several small fixes have accumulated and you need to understand what changed in the workflow.

Provide the actual records rather than asking the assistant to remember previous events. It should identify resolved issues, outstanding questions, and any changes that affect existing regression checks.

The summary should never invent a successful test run or imply that a particular incident is resolved without supporting evidence. If a test was suggested but not executed, the record should say so.

Keep maintenance proportionate and revisit old assumptions

A personal automation should become easier to maintain as your test history grows, not buried under documents that no one reads. Keep the workflow small enough that you will actually use it.

Review the oldest regression tests when the project undergoes major changes. Some tests may still protect essential behavior. Others may refer to requirements that have changed deliberately.

If you retire a test, record why it is no longer relevant. If you change a task identity rule or reminder schedule, make the new contract explicit before updating the related assertions.

A lightweight maintenance routine can also include occasional checks of dependencies, expired credentials, and scheduled execution settings. These concerns are not always regression tests, but they can explain why an automation changes behavior even when its source code remains stable.

Frequently Asked Questions

Conclusion: make every bug fix easier to trust next time

Personal automations are meant to remove repetitive work, but an unreliable automation can create a different kind of burden. When every failure starts a new AI conversation from scratch, the same assumptions may be rediscovered again and again.

A maintainable workflow keeps the important facts. It preserves the failing input, states the intended behavior, records the accepted repair, and runs a regression check when later changes are introduced. AI can help with analysis and test construction, while the actual code and execution results remain the source of verification.

The duplicate-reminder example shows why precise boundaries matter. Removing repeated task rows in one import batch is a useful improvement, but it does not guarantee that the reminder will be delivered only once across retries or restarted jobs. Keeping those issues separate makes the maintenance process more accurate.

Begin with the next meaningful error in an automation you use regularly. Give it an incident ID, save a small reproduction, and create one test that would fail if the same problem returned. You do not need to build an elaborate testing platform. You need a repeatable way to turn a repaired failure into a rule that the next version must respect.