Announcing Patchwork

About a month ago, we joked about shutting down - the date should have been a giveaway! As morbid as that post was, we are excited to share that patched is still going strong and is now open-source with the release of Patchwork.

‍

What is Patchwork?

Patchwork provides open-source AI workflows for chore automation using large language models. It is an all-python framework that allows dev teams to orchestrate custom workflows - called ‘patchflows’ - using a combination of reusable steps and prompts. These patchflows can be used to fix vulnerabilities, upgrade dependencies, generate documentation, and more.

We always envisioned Patched as a platform that enabled developers to automate and accelerate tasks that they find tedious, time-consuming, or disruptive. Github Copliot and other coding assistants do a great job of helping developers in the IDE as they write code - also known as the dev inner loop. But the dev outer loop requires multiple steps and tools to be executed together with appropriate context and developer input. This is the set of tasks that can be automated using Patchwork. Let’s use an example to dig deeper.

‍

Fixing vulnerabilities with Patchwork

Patching code vulnerabilities with LLMs sounds simple. Take the code, throw it into ChatGPT, ask it to make it secure, and copy the fixed code back. Right?

In the real world, dev teams are constantly being bombarded with scan results from security tools that are notorious for their high false-positive rate. As a result, Devs have to ‘triage’ these results by considering the application context alongside their own judgment. With the false positives removed, a dev then creates a patch for the vulnerability - something that isn’t always straightforward and requires security training. But the job isn’t done yet. The patch needs to be validated to ensure that it is does not break existing functionality and is compatible with the larger codebase before it can be merged.

We created the AutoFix patchflow to automate this entire process in a step-wise manner - integrating with source control, scanning tools, and LLM endpoints. Not only can LLMs be used for generating fixes, they also do a good job of triaging and ensuring compatibility. How well? The chart below shows the performance (and costs) across different models using the AutoFix patchflow with default inputs and prompts.

‍

No More (Dev) Chores

We realized that the list of repetitive, time-consuming tasks faced by developers extends beyond fixing vulnerabilities. To this effect, we have provided 4 additional patchflows for automating common development chores:

• PRReview: On PR creation, extract code diff, summarize changes, and comment on PR.
• GenerateREADME: Create a README markdown file for a given folder, to add documentation to your repository.
• [Experimental] DependencyUpgrade: Update your dependencies from vulnerable to fixed versions.
• [Experimental] ResolveIssue: Identify the files in your repository that need to be updated to resolve an issue (or bug) and create a PR to fix it.

And if these don’t cover a task that you would like to automate, don’t worry. You can create your own patchflows seamlessly by following our documentation - or soon from the patched app UI!

‍

Why Open Source?

Open-sourcing patchwork was a no-brainer. Both from a pragmatic, and idealistic perspective.

From a pragmatic standpoint, the current generation of LLMs are too unpredictable when left to their own devices to figure out an execution path given a high-level instruction. Additionally, even the most narrow, well-equipped AI agents struggle to reliably complete the same task in different environments. We believe that the first generation of truly usable AI agents will need users to define and customize their behavior without constraints. And open source is the best way to enable that. 

From an idealistic perspective, our team has a long history of working with open-source and being part of the community. The open-source ethos of collaboration, transparency, and open innovation resonates strongly with us - and we are excited to give a little bit back to the community that we have gained so much from.

‍

The Path Ahead

We are looking to expand our list of steps and patchflows to unlock more automation possibilities, together with some performance and usability improvements in the coming weeks. We are also planning an updated release of the patched app in June, complete with the ability to create patchflows with a visual editor. If you’d like an early sneak peak into this new era of Patched, drop us a line and we’d love to show you more!