Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Base URL of the deployed Modal gateway, e.g.
# https://<workspace>--cutout-generator-api.modal.run
NEXT_PUBLIC_API_URL=
3 changes: 0 additions & 3 deletions .eslintignore

This file was deleted.

17 changes: 0 additions & 17 deletions .eslintrc.cjs

This file was deleted.

38 changes: 38 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: CI

on:
push:
branches: [master, main, develop]
pull_request:

jobs:
checks:
name: Lint, typecheck, test & build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm

- name: Install dependencies
run: npm ci

- name: Prettier
run: npm run format:check

- name: Lint
run: npm run lint

- name: Typecheck
run: npm run typecheck

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Generate Next route types before running tsc

In the new CI workflow, npm run typecheck runs before any next build/dev step, but this commit also tracks next-env.d.ts with import './.next/types/routes.d.ts'; while .next is gitignored and only generated by Next. On a clean GitHub runner this ordering makes tsc --noEmit fail before the build step can create the route-types file; either stop committing that generated import or run the Next type generation/build before the standalone typecheck.

Useful? React with 👍 / 👎.


- name: Test
run: npm run test

- name: Build
run: npm run build
env:
NEXT_PUBLIC_API_URL: https://example.modal.run
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -131,4 +131,6 @@ dist

.DS_Store

.env*
.env*
# keep the env template
!.env.example
1 change: 0 additions & 1 deletion .prettierignore

This file was deleted.

5 changes: 5 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"singleQuote": true,
"printWidth": 100,
"trailingComma": "es5"
}
1 change: 0 additions & 1 deletion .prettierrc.cjs

This file was deleted.

37 changes: 0 additions & 37 deletions .storybook/main.ts

This file was deleted.

25 changes: 0 additions & 25 deletions .storybook/preview.tsx

This file was deleted.

893 changes: 0 additions & 893 deletions .yarn/releases/yarn-4.0.1.cjs

This file was deleted.

7 changes: 0 additions & 7 deletions .yarnrc.yml

This file was deleted.

94 changes: 34 additions & 60 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,76 +1,50 @@
# AI Image Cutout Maker
# AI Cutout Maker — Frontend

AI Image Cutout Maker is a project that uses artificial intelligence to automatically create cutouts from images. This project is designed to simplify the process of creating cutouts, which can be a time-consuming task if done manually.
Next.js 16 (App Router) frontend for the [AI Image Cutout Maker backend](https://github.com/OriginalByteMe/AI_Image_cutout_maker), powered by SAM 3 on Modal.

This project utilizes the power of Segment Anything and Grounding Dino AI models to detect subjects in an image and cut them out. These models are hosted on Modal, which allows us to leverage GPU acceleration for faster and more efficient processing.
Describe what you want cut out of your images and videos — every matching subject comes back as a transparent PNG, a WhatsApp-ready sticker (512x512 WebP), or a transparent-background WebM for videos.

The cutouts are then stored in an Amazon S3 bucket, providing a scalable and secure storage solution. This setup allows us to handle large volumes of images and serve them quickly and efficiently.
## Pages

## Project Structure
- `/` — landing page
- `/studio` — bulk image cutouts: drop many images, add subject prompts, tune the confidence threshold, download individual PNGs/stickers or everything as a zip
- `/video` — video cutouts: upload a clip, describe a subject, poll the async job, preview & download the alpha WebM

The project is structured as follows:
## Stack

- `app/`: This directory contains the main application code.
- `cutout.py`: This script handles the process of creating cutouts from images using the Segment Anything and Grounding Dino AI models.
- `dino.py`: This script is responsible for interacting with the Grounding Dino AI model.
- `segment.py`: This script is used for interacting with the Segment Anything AI model.
- `s3_handler.py`: This script handles interactions with Amazon S3, such as uploading and downloading images.
- `grounded_cutouts.py`: This script ...
- **Next.js 16** (App Router) + **React 19**
- **Mantine 9** for UI, **@tabler/icons-react** for icons
- **TanStack Query 5** for mutations/polling
- **Zod 4** validates every API response at runtime (`lib/api.ts` is the single typed API client)
- **Vitest + Testing Library** for tests, **ESLint 9 (flat config) + Prettier** for hygiene

- `.venv/`: This directory contains the virtual environment for the project.
## Getting started

- `modal_utils/`: This directory contains utility functions used throughout the project.

- `grpc_utils.py`: This script handles the gRPC connections in the project.


## Purpose of the Project

The purpose of this project is to automate the process of creating cutouts from images. By using artificial intelligence, we can create accurate cutouts much faster than would be possible manually. This can be useful in a variety of applications, such as graphic design, image editing, and more. There is also the social media aspect of creating stickers out of items cut out of an image, which is a popular trend on social media platforms such as Instagram and TikTok.

## How to Use

As mentioned above, this project is being hosted through Modal, this also does mean that its using the modal API. This means that you will need to have a modal account and have the modal CLI installed. You can find instructions on how to do this [here](https://docs.modal.ai/docs/getting-started). I'am planning on dockerizing this project as well down the line so that it can be used without the modal CLI.


## Workflow diagriam for how BE processes and returns values
```mermaid
sequenceDiagram
actor User
User ->> CutoutFE: Upload Image
CutoutFE ->> S3: Call to push image to bucket
CutoutFE ->> CutoutBE: Call to create cutout
CutoutBE ->>+ Modal: Spin up instance with container
Modal ->>- CutoutBE: Allow that container to be used by CutoutBE URL
CutoutBE ->>+ S3: Download image
S3 -->>- CutoutBE: Send Image back
Note over CutoutBE: Processes cutouts + other diagrams
CutoutBE -->> S3: Upload cutouts to bucket
Note over CutoutBE: Generates Pre-signed URL's of processed files
Note over CutoutFE: If need be FE can also create list of presigned urls via s3
CutoutBE -->>+ CutoutFE: Return list of presigned urls
CutoutFE -->>+ User: Display processed images
User ->> S3: Download images via url
```bash
npm install
cp .env.example .env.local # set NEXT_PUBLIC_API_URL to your Modal gateway URL
npm run dev
```

`NEXT_PUBLIC_API_URL` should point at the deployed Modal gateway, e.g.
`https://<workspace>--cutout-generator-api.modal.run`.

## To-Do

Here are some tasks that are on our roadmap:

- Dockerize the project: We plan to create a Dockerfile for this project to make it easier to set up and run in any environment.
- API Documentation: We will be writing comprehensive API documentation to make it easier for developers to understand and use our API.
- Improve error handling: We aim to improve our error handling to make our API more robust and reliable.
- Add more AI models: We are planning to integrate more AI models to improve the accuracy and versatility of our image cutout creation.
- Optimize performance: We will be working on optimizing the performance of our API, particularly in terms of processing speed and resource usage.

Please note that this is not an exhaustive list and the roadmap may change based on project needs and priorities.
## Scripts

## Contributing
| Script | What it does |
|---|---|
| `npm run dev` | dev server |
| `npm run build` / `start` | production build / serve |
| `npm run lint` | ESLint |
| `npm run typecheck` | `tsc --noEmit` |
| `npm run test` | Vitest |
| `npm run format` / `format:check` | Prettier |
| `npm run check` | format check + lint + typecheck + tests (what CI runs) |

This is a personal project so it won't really be geared to any contributions, but feel free to fork the repository and make any changes you want. If you have any questions, feel free to reach out to me at my [email](mailto:noahrijkaard@gmail.com)
## CI

## License
`.github/workflows/ci.yml` runs format check, lint, typecheck, tests and a production build on every push/PR.

This project is licensed under the terms of the [MIT License](LICENSE).
## Note on zip downloads

"Download all as zip" fetches the presigned S3 URLs from the browser, so the S3 bucket needs a CORS rule allowing `GET` from your frontend origin. Single-file downloads and previews work without it.
23 changes: 23 additions & 0 deletions app/globals.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/* Checkerboard backdrop that makes transparency visible. */
.checkerboard {
background-image: conic-gradient(
rgba(128, 128, 128, 0.25) 90deg,
transparent 90deg 180deg,
rgba(128, 128, 128, 0.25) 180deg 270deg,
transparent 270deg
);
background-size: 16px 16px;
border-radius: var(--mantine-radius-md);
}

.cutout-preview {
width: 100%;
height: 180px;
object-fit: contain;
display: block;
}

.video-preview {
width: 100%;
max-height: 420px;
}
32 changes: 32 additions & 0 deletions app/layout.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import '@mantine/core/styles.css';
import '@mantine/dropzone/styles.css';
import '@mantine/notifications/styles.css';
import './globals.css';

import type { Metadata } from 'next';
import { ColorSchemeScript, mantineHtmlProps } from '@mantine/core';
import { Providers } from './providers';
import { Header } from '@/components/Header';

export const metadata: Metadata = {
title: 'AI Cutout Maker',
description:
'Describe what you want cut out of your images and videos — SAM 3 does the rest. Transparent PNGs, WhatsApp stickers and alpha videos.',
};

export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="en" {...mantineHtmlProps}>
<head>
<ColorSchemeScript defaultColorScheme="auto" />
<link rel="icon" href="/favicon.svg" />
</head>
<body>
<Providers>
<Header />
<main>{children}</main>
</Providers>
</body>
</html>
);
}
Loading
Loading