This is a FOSS tool that analyzes emails using a combination of open source tools and free-to-use third-party APIs.
Notable integrations include...
- 4 Cores CPU
- 8GB Memory
- 32GB Disk
- Stable 50mbps+ network connection
- Email parsing (headers, body, attachments)
- Spam and phishing detection
- Malware scanning for attachments
- Sender IP and origin analysis
- Risk scoring using rule based detection
- Email comparison using a Diff Checker
- FastAPI Backend
- Central orchestration layer
- Handles uploads, processing, and responses
- Rspamd
- Spam filtering and scoring engine
- ClamAV
- Malware and virus scanning for attachments
- Redis
- Temporary data storage for processing efficiency
- Docker
- All services run in isolated containers on a private docker network
- Only the API endpoint is externally accessible
- User uploads .eml file
- API processes the request
- Email is parsed into:
- Headers
- Plain text body
- HTML body
- Attachments
- Data is sent to:
- Rspamd for spam/phishing analysis
- ClamAV for malware scanning
- Sender IPs are parsed and traced to their origin
- Results are aggregated
- Risk score and analysis are returned to the user
We've made running your own instance of our API as simple as possible. Just follow the steps below.
-
Install docker on the same host you will be running the API. Here are some recommended installation guides.
- Ubuntu - Digital Ocean
-
Clone the repository.
git clone https://github.com/Sentinel-Systems-Drexel/backend.git
-
For map functionality, create a Mapbox account and add your API token as an environment variable named
MAPBOX_TOKEN(.env file in project root). -
Customize settings in the compose file (
docker-compose.yml)
CACHE_MAPS (default: false)
Choose whether or not the system should cache map images generated by Mapbox. This uses a significant amount of disk space, and is only recommended if you are reaching the limits of the Mapbox API.
DATA_RETENTION_MINUTES (default: 0)
The amount of time in minutes that email analysis data is saved for. Value of 0 indicates that analysis data is stored indefinitely. Changing this setting will affect files that were created before the change was made (0 -> 10 minutes, ALL analysis files older than 10 minutes will be deleted).
DATA_INDEXING_MINUTES (default: 5)
The frequency in minutes at which analysis files are indexed for deletion.
CORS_ALLOW_ORIGINS (default: *)
Limits which origins can access your endpoint. Allows all origins by default.
E.g. CORS_ALLOW_ORIGINS=https://example.com/tool, https://tool.example.com, https://ui-fork.example2.com
In the above example, all listed origins will be able to make requests to the endpoint.
CORS_ALLOW_CREDENTIALS (default: False)
Allow credentialed requests to the endpoint. Disabled by default.
CORS_ALLOW_METHODS (default: GET, POST)
Customize allowed methods for incoming requests. Only GET and POST are enabled by default.
CORS_ALLOW_HEADERS (default: Options, Content-Type)
Customize allowed headers for incoming requests. Only Options and Content-Type are allowed by default.
- Save your changes if you made any. Then start the cluster.
sudo docker compose up -d --build
- The API should be reachable at
http://<host_address>:8000. E.g.http://127.0.0.1:8000,http://localhost:8000,http://192.168.1.26:8000.
This is the procedure for updating settings without wiping data and fully rebuilding all containers.
- Update settings in the compose or env file.
- Save your changes.
- Run the following command to rebuild the API with your changes.
docker compose up -d --build api --remove-orphans
Email analysis files and map cache files are stored in host-mounted directories so they persist across container restarts and rebuilds. Email analysis files are set to persist indefinitely by default. This value (DATA_RETENTION_MINUTES) can be modified in the compose file.
- Host paths:
./email-analysis-data,./maps-cache,./logs - Container paths:
/data/email-analysis,/data/maps_cache,/data/logs