Creating an API documentation of a website from scratch

Description

Reverse engineering an API assists in discovering potential endpoints for evaluating its functionality, a critical aspect of testing and analysis. When access to official documentation or the API’s original codebase is unavailable, these methods enable the manual creation of documentation.

Environment and Tools Used

Application

Here I am using Parabank by Parasoft. Parabank is a web application used to demonstrate web, API testing. The application is available in dockerhub. The best way to run this is to pull it from docker and host it locally. We can also find live instances of this website on the web.

Docker basic commands

    sudo docker pull 
    sudo docker ps -a 
    sudo docker stop <containerid>
    sudo docker rm <containerid>

Postman

mitmweb

mitmproxy2swagger

pip install  mitmproxy2swagger

Methods

Using Postman built-in proxy

Utilizing Postman’s built-in proxy feature facilitates the creation of API collections. This functionality allows seamless capture of API requests and responses, simplifying the process of assembling comprehensive collections within the Postman environment.

On starting Postman, we could see capture requests in the bottom right corner

postman1 Click on Enable proxy and enter the port number - Enable proxy

postman2

Enter the domain of the URL in additional configuration and start capturing. postman3

Set the proxy to the port set in Postman and start clicking around the webpage. Use it as it was meant to be used. postman4

We could see the session is being captured in Postman. Once done stop and we could get our collection. The only thing is we need to manually sort the sessions parsed in postman. postman5

MITM Web and mitmproxy2swagger

MITM web is a web-based user interface for Mitmproxy. This intercept and inspect HTTP and HTTPS traffic in real time and helps us understand how information flows between clients and servers, identify potential security vulnerabilities, debug network issues, and test the behavior of applications under different network conditions.

To start mitmweb, just type mitmweb in the terminal. The proxy uses 8080 as the default. we can change it using the -p switch.

mitmweb -p 8888

once it starts, the web interface automatically loads in our browser. Set the proxy to go through the defined port. Again browse the application and go through all the pages, sign up, log in, create an account, take a loan, change the address, transfer money, and so on. mitmweb1 mitmweb2 Our proxy would be filled with sessions. Type in the domain/IP of our application in the search tab to sort out the sessions related to our relevant website. mitmweb3 Save the filtered traffic. mitmweb4

We could see the flow file being downloaded locally. mitmweb5 Now onto creating the swagger file from the flows using mitmproxy2swagger

sudo mitmproxy2swagger -i flows -o spec.yml -p http://your-ip-address:8080/parabank -f flow --examples

This tries to filter only the API-based requests, but we need to manually check and see whether any API-specific endpoints are being ignored.

Edit the created spec.yml file as sudo and remove the ignore: tag from the particular requests. We can even change the title. mitmweb6

Rerun the command to create our spec.yml file again

sudo mitmproxy2swagger -i flows -o spec.yml -p http://your-ip-address:8080/parabank -f flow --examples

Now import the spec.yml file to the online swagger editor. File > Import file mitmweb7

Voila !! Much more uncluttered swagger file for our purpose

We can even import it in Postman using import and select the spec.yml file. mitmweb8 mitmweb9

This gives us a much better collection to work with.

Through these approaches, we can generate our API documentation for an API lacking public documentation while testing it in a black-box scenario. This method may not unveil all endpoints initially, but it serves as a beneficial starting point. For instance, the Parabank application offers the API swagger file within its system. Comparing this internal documentation allows us to validate whether we’ve captured all available endpoints.

Location to Parabank swagger file - http://your-ip-address:8080/parabank/services/bank/swagger.yaml

Happy Hacking!!