0xb00b5/firegex-traffic-viewer
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

README updates

Browse Source
This commit is contained in:
DomySh
2022-06-24 22:44:45 +02:00
parent 390ad7a2fa
commit 186ac74065
11 changed files with 334 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.dockerignore
View File

@@ -20,6 +20,8 @@ docker-compose.yml
/backend/proxy/proxy
/docs/**
# misc
**/.DS_Store
**/.env.local

217
README.md
View File

@@ -1,192 +1,53 @@
# **WORK IN PROGRESS**
![Firegex Logo](docs/header-logo.png)
# Firegex-API Documentation
### This is a short description of the API
# Firegex
#
# Documentation
## Index
## What is Firegex?
Firegex is a reverse-proxy application firewall created for CTF Attack-Defence competitions that has the aim to limit of totally deny TCP malicious traffic throught the use of regex filters.
- [General stats](#get-apigeneral-stats)
- [List services](#get-apiservices)
- [Service info](#get-apiserviceserv)
- [Stop service](#get-apiserviceservstop)
- [Start service](#get-apiserviceservstart)
- [Pause service](#get-apiserviceservpause)
- [Delete service](#get-apiserviceservdelete)
- [Terminate service](#get-apiserviceservterminate)
- [Regenerate public port](#get-apiserviceservregen-port)
- [Service regexes](#get-apiserviceservregexes)
- [Regex info](#get-apiregexregexid)
- [Delete regex](#get-apiregexregexiddelete)
- [Add regex](#post-apiregexesadd)
- [Add service](#post-apiservicesadd)
![Firegex Network scheme](docs/FiregexWorking.png)
#
#
## **GET** **```/api/general-stats```**
### Server response:
```json
{
"services": <total number of services>,
"closed": <total number of rejected packets>,
"regex": <total number of regexes>
}
```
Firegex don't replace the network firewall, but works together with it.
#
## **GET** **```/api/services```**
### Server response:
```json
[
{
"id": <service_id>,
"status": <service status>,
"public_port": <public port>,
"internal_port": <internal port>,
"n_packets": <number of rejected packets>,
"n_regex": <number of regexes>
},
{
// Another service
}
]
```
## How it works?
#
## **GET** **```/api/service/<serv>```**
### Server response:
```json
{
"id": <service_id>,
"status": <service status>,
"public_port": <public port>,
"internal_port": <internal port>,
"n_packets": <number of rejected packets>,
"n_regex": <number of regexes>
}
```
When you start firegex, the first step is to create the services that it has to proxy.
The name of the service and the port where is expected to be hosted are the only things required. For each service, a random intermediate port is generated.
You can start all the services you have created on the proxy. If the port it's free, firegex will blind it, and forward the connection to the random port generated, it's expected that the real service will start at that port. If the port is already blinded, firegex will keep tring to blind that port until it will success. This allow to reduce the down time during the transition from the old port to the new port used by the real service. When the real service will have changed port, the proxy will automatically handle the port and start proxying the real with the port setted on the firewall.
#
## **GET** **```/api/service/<serv>/stop```**
### Server response:
```json
{
"status": "ok"
}
```
Remember to start the service with the intermediate port publishing this only on localhost network or blocking the public access using a network firewall.
#
## **GET** **```/api/service/<serv>/start```**
### Server response:
```json
{
"status": "ok"
}
```
Now, you can enter in the service detail, and manage the filtering rules to use for each services. Thet service can be in 4 different states:
- START: The proxy is running and it's filtering all tcp packets using the regex added
- PAUSE: The proxy is running, but it's not filtering the packets, it's only keeping the service active continuing forwarding the packets
- WAIT: The proxy is not running, but it's waiting until the port to blind will be free, after that the proxy will go in PAUSE or START mode according to what requested previously
- STOP: The proxy is not running.
#
## **GET** **```/api/service/<serv>/delete```**
### Server response:
```json
{
"status": "ok"
}
```
You can change the status clicking the button provided in the frontend. If you want to add a regex or delete a regex you can add or remove it, and if the service is in START mode, the regex changes will have an immediate effect on the proxy that will start following the new ruleset.
#
## **GET** **```/api/service/<serv>/pause```**
### Server response:
```json
{
"status": "ok"
}
```
## Documentation
#
## **GET** **```/api/service/<serv>/regen-port```**
### Server response:
```json
{
"status": "ok"
}
```
Find the documentation of the backend and of the frontend in the related README files
#
## **GET** **```/api/service/<serv>/regexes```**
### Server response:
```json
[
{
"id": <regex id>,
"service_id": <service_id>,
"regex": <base64 encoded regex>,
"is_blacklist": <true|false>,
"is_case_sensitive": <true|false>,
"n_packets": <number of blocked packets>,
"mode": <"C"|"S"|"B"> // Client to server, server to client or both
},
{
// Another regex
}
]
```
- [Frontend (React)](frontend/README.md)
- [Backend (Flask + C++)](backend/README.md)
#
## **GET** **```/api/regex/<regex_id>```**
### Server response:
```json
{
"id": <regex id>,
"service_id": <service_id>,
"regex": <base64 encoded regex>,
"is_blacklist": <true|false>,
"is_case_sensitive": <true|false>,
"n_packets": <number of blocked packets>,
"mode": <"C"|"S"|"B"> // Client to server, server to client or both
}
```
![Firegex Working Scheme](docs/FiregexInternals.png)
#
## **GET** **```/api/regex/<regex_id>/delete```**
### Server response:
```json
{
"status": "ok"
}
```
#
## **POST** **```/api/regexes/add```**
### Client request:
```json
{
"service_id": <service_id>,
"regex": <base64 encoded regex>,
"is_blacklist": <true|false>,
"is_case_sensitive": <true|false>,
"mode": <"C"|"S"|"B"> // Client to server, server to client or both
}
```
### Server response:
```json
{
"status": "ok"
}
```
#
## **POST** **```/api/services/add```**
### Client request:
```json
{
"name": <the id used to identify the service>,
"port": <the internal port>
}
```
### Server response:
```json
{
"status": "ok"
}
```
### Main Points of Firegex
#### 1. Efficiency
Firegex should not slow down the traffic on the network. For this the core of firegex it's a c++ binary file.
1. The proxy itself is build with a binary c++ file that uses the boost library, well-known for it's stability and efficiency.
2. The proxy works thanks to async io calls, granting great efficiency and minimum time loss
3. The filter is done by the binary file using the regex std c++ library (in the firsts versions of firegex, the boost::regex library was used, but after some tests, and the rising of some problems with this library, we passed to the std lib also looking at the similar efficiency and more stability with the same tests we done)
#### 2. Availability
Firegex **must** not become a problem for the SLA points!
This means that firegex is projected to avoid any possibility to have the service down. We know that passing all the traffic through firegex, means also that if it fails, all services go down. It's for this that firegex implements different logics to avoid this.
1. Every reverse proxy is isolated from each other, avoiding the crash of all the proxies started by firegex
2. The proxy is a binary program written in C, started as an indipendent process with indipendent memory, and uses boost lib for the connection and the std lib for checking the regex for each packet
3. If a regex fails for whatever reason, the proxy remove this from the filter list and continue to forward the packets like it did't exist.
4. If the firewall is restarted, at the startup it try to rebuild the previous status of proxies
5. The firewall interface it's protected by a password. No one excepts your team must have access to firegex, this can be really really dangerous!
6. If a regex makes trouble, you can delete it (this have an instant effect on the proxy), or put the service in pause (call also Transparent mode), this will deactivate all the filters from the proxy, but still continue to publish the service on the right port
7. Every status change (except if you decide to stop the proxy) that you made to the service, and so to the proxy is instantaneous and done with 0 down time. The proxy is **never** restarted, it's configuration changes during runtime

283
backend/README.md Normal file
View File

@@ -0,0 +1,283 @@
# Firegex backend
## [GO BACK](../README.md)
The backend of firegex is written with flusk, runned by uwsgi. The aim of the backend is to manage all the requests from the react front-end and manage also the proxy processes of the entire firewall. For this reason the backend is divided into 2 main parts:
![Firegex Working Scheme](../docs/FiregexInternals.png)
## The Flask module
This module recieve the requests and manage these doing some queries to the SQLite DB, or sending signals and request to the Proxy-Manager module.
## The Proxy Manager
The proxy manager is started by the flask backend, but is indipendent from that. The proxy manager offers the api to the backend for abstract the managment of the proxies needed to make the firewall working. This module use also the SQLite db to syncronize its data about the packet filtered, the status of services and the regex added/removed.
For each service created this module create a Thread that manage all the complexity about make working the proxy, updating it's status from the database.
![Proxy Manager Wrapping](../docs/ProxyManagerWrapping.png)
Firegex is reliable thanks to the fact that it's proxy it's not a python proxy, but it it's wrote in c++ using boost lib. This allow to have an high efficency proxy and high efficency regex filter match. A python wrapping and ubstraction module allow the use of this binary, that it's not been thought to be easy to use for humans. The wrapper allow to have from the binary all the needed statistics, and update it's status and it's regex without any downtime of the service: the changes are catched and executed during the execution of the proxy.
## [GO BACK](../README.md)
# API Documentation
## Index
### Platform API
- [Platform and session status](#get-apistatus)
- [Login](#post-apilogin-only-in-run-mode)
- [Logout](#get-apilogout)
- [Change Password](#post-apichange-password-only-in-run-mode--login-required)
- [Set Password](#post-apiset-password-only-in-init-mode)
### Data API
- Info API
- [General stats](#get-apigeneral-stats-login-required)
- Services:
- [Add service](#post-apiservicesadd-login-required)
- [Delete service](#get-apiserviceservdelete-login-required)
- [Service info](#get-apiserviceserv-login-required)
- [List services](#get-apiservices-login-required)
- Regexes:
- [Add regex](#post-apiregexesadd-login-required)
- [Delete regex](#get-apiregexregexiddelete-login-required)
- [Regex info](#get-apiregexregexid-login-required)
- [Service regexes](#get-apiserviceservregexes-login-required)
- Proxy Managment API
- [Stop service](#get-apiserviceservstop-login-required)
- [Start service](#get-apiserviceservstart-login-required)
- [Pause service](#get-apiserviceservpause-login-required)
- [Regenerate public port](#get-apiserviceservregen-port-login-required)
#
## **GET** **```/api/status```**
### Server response:
```jsonc
{
"status": <"run"|"init">,
"loggined": <is true if the request is have a loggined session cookie>
}
```
#
## **POST** **```/api/login```** `ONLY IN RUN MODE`
### Client request:
```jsonc
{
"password": <the firewall password>,
}
```
### Server response:
```jsonc
{
"status": <"ok"/"Wrong password!"/"Cannot insert an empty password!">
}
```
#
## **GET** **```/api/logout```**
### Server response:
```jsonc
{
"status": "ok",
}
```
#
## **POST** **```/api/change-password```** `ONLY IN RUN MODE` + `LOGIN REQUIRED`
### Client request:
```jsonc
{
"password": <the firewall password>,
"expire": <if true, the app secret key will be changed causing the logout of every session>
}
```
### Server response:
```jsonc
{
"status": <"ok"/"Cannot insert an empty password!">
}
```
#
## **POST** **```/api/set-password```** `ONLY IN INIT MODE`
### Client request:
```jsonc
{
"password": <the firewall password>,
}
```
### Server response:
```jsonc
{
"status": <"ok"/"Cannot insert an empty password!">
}
```
#
## **GET** **```/api/general-stats```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"services": <total number of services>,
"closed": <total number of rejected packets>,
"regex": <total number of regexes>
}
```
#
## **GET** **```/api/services```** `LOGIN REQUIRED`
### Server response:
```jsonc
[
{
"id": <service_id>,
"status": <service status>,
"public_port": <public port>,
"internal_port": <internal port>,
"n_packets": <number of rejected packets>,
"n_regex": <number of regexes>
},
{
// Another service
}
]
```
#
## **GET** **```/api/service/<serv>```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"id": <service_id>,
"status": <service status>,
"public_port": <public port>,
"internal_port": <internal port>,
"n_packets": <number of rejected packets>,
"n_regex": <number of regexes>
}
```
#
## **GET** **```/api/service/<serv>/stop```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"status": "ok"
}
```
#
## **GET** **```/api/service/<serv>/start```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"status": "ok"
}
```
#
## **GET** **```/api/service/<serv>/delete```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"status": "ok"
}
```
#
## **GET** **```/api/service/<serv>/pause```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"status": "ok"
}
```
#
## **GET** **```/api/service/<serv>/regen-port```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"status": "ok"
}
```
#
## **GET** **```/api/service/<serv>/regexes```** `LOGIN REQUIRED`
### Server response:
```jsonc
[
{
"id": <regex id>,
"service_id": <service_id>,
"regex": <base64 encoded regex>,
"is_blacklist": <true|false>,
"is_case_sensitive": <true|false>,
"n_packets": <number of blocked packets>,
"mode": <"C"|"S"|"B"> // Client to server, server to client or both
},
{
// Another regex
}
]
```
#
## **GET** **```/api/regex/<regex_id>```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"id": <regex id>,
"service_id": <service_id>,
"regex": <base64 encoded regex>,
"is_blacklist": <true|false>,
"is_case_sensitive": <true|false>,
"n_packets": <number of blocked packets>,
"mode": <"C"|"S"|"B"> // Client to server, server to client or both
}
```
#
## **GET** **```/api/regex/<regex_id>/delete```** `LOGIN REQUIRED`
### Server response:
```jsonc
{
"status": "ok"
}
```
#
## **POST** **```/api/regexes/add```** `LOGIN REQUIRED`
### Client request:
```jsonc
{
"service_id": <service_id>,
"regex": <base64 encoded regex>,
"is_blacklist": <true|false>,
"is_case_sensitive": <true|false>,
"mode": <"C"|"S"|"B"> // Client to server, server to client or both
}
```
### Server response:
```jsonc
{
"status": <"ok"|"Invalid regex"|"An identical regex already exists">
}
```
#
## **POST** **```/api/services/add```** `LOGIN REQUIRED`
### Client request:
```jsonc
{
"name": <the id used to identify the service>,
"port": <the internal port>
}
```
### Server response:
```jsonc
{
"status": <"ok"|"Name or/and port of the service has been already assigned to another service">
}
```
## [GO BACK](../README.md)

6
backend/app.py
View File

@@ -64,7 +64,7 @@ def get_status():
@app.route("/api/login", methods = ['POST'])
def login():
if not conf.get("password"): return abort(404)
if app.config["STATUS"] != "run": return abort(404)
req = request.get_json(force = True)
try:
@@ -95,6 +95,7 @@ def logout():
@app.route('/api/change-password', methods = ['POST'])
@login_required
def change_password():
if app.config["STATUS"] != "run": return abort(404)
req = request.get_json(force = True)
try:
@@ -122,9 +123,8 @@ def change_password():
@app.route('/api/set-password', methods = ['POST'])
def set_password():
if conf.get("password"): return abort(404)
if app.config["STATUS"] != "init": return abort(404)
req = request.get_json(force = True)
try:
validate(
instance=req,

0
Dockerfile.fullcompile → docs/Dockerfile.fullcompile
View File

BIN
docs/FiregexInternals.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 164 KiB

BIN
docs/FiregexWorking.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 196 KiB

BIN
docs/Firegex_Screenshot.jpg Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 47 KiB

BIN
docs/ProxyManagerWrapping.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 60 KiB

BIN
docs/header-logo.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 71 KiB

49
frontend/README.md
View File

@@ -1,46 +1,11 @@
# Getting Started with Create React App
# Firegex Frontend
This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
![Firegex Frontend](../docs/Firegex_Screenshot.jpg)
## Available Scripts
The frontend of firegex is written using create-react-app.
The routing of the pages is managed by react-router 6 and the graphics is mainly managed by the framework [Mantine](https://mantine.dev) and icons provided by [React Icons](https://react-icons.github.io/react-icons/).
The style of the page is written with [sass](https://sass-lang.com/).
In the project directory, you can run:
The page is auto-updated by a global timeout that raise an event triggering all parts of the app that requires fetch updated data from the backend, the update of these data.
### `npm start`
Runs the app in the development mode.\
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
The page will reload if you make edits.\
You will also see any lint errors in the console.
### `npm test`
Launches the test runner in the interactive watch mode.\
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
### `npm run build`
Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.
The build is minified and the filenames include the hashes.\
Your app is ready to be deployed!
See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
### `npm run eject`
**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
## Learn More
You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
To learn React, check out the [React documentation](https://reactjs.org/).
## [GO BACK](../README.md)