The cat under the hood. Part 2

Hello! In the last part, we talked about existing solutions for "orchestrating" the parallel tasks of "hash cracking" on hashcat.

In this part, we continue to talk about the Hashtopolis project. Consider preparing the components of the web environment on the server. Let's analyze the main parameters of the server and Hashtopolis agent.

Installing and Configuring Hashtopolis

For its work, the Hashtopolis server wants the following components from you:

• Apache2 / nginx
• 64-bit PHP 7.0+ and modules: php-gd modules, pear, php-curl
• MySQL 5.x + KM with full access to the application database

Hashtopolis has nginx support, for which you need to configure access blocking for all "/ hashtopolis" subdirectories containing the .htaccess file. In this article, we are considering installing Hashtopolis with the Apache2 web server (therefore LAMP).

You can also use XAMPP to get acquainted with the project , on version 5.6.40 everything “took off” without problems. The material in this article is useful in configuring Hashtopolis on XAMPP.

Under the spoiler, a little about the Hocktopolis Docker image:

LAMP Installation

The version of the project that we used: Hashtopolis 0.10.1

For the Hashtopolis server to work, the host with the following configuration was enough for us:

• 1 CPU
• 1GB RAM
• 25 GB disk

We recommend installing phpMyAdmin, Adminer, or another analog for working with the DBMS. Thus, in case of problems with the Hashtopolis functionality, Troubleshoot can be performed without leaving the browser.

In the process of installing software on our LAMP server, we used the instructions from the Wiki of the Hashtopolis project (https://github.com/s3inlc/hashtopolis/wiki/Server-Prerequisites):

1. sudo apt update && sudo apt upgrade
2. sudo apt install mysql-server
3. sudo apt install apache2
4. sudo apt install libapache2-mod-php php-mysql php php-gd php-pear php-curl
5. sudo apt install git
6. sudo apt install phpmyadmin

If the interactive configuration wizard was not launched during the installation of mysql packages, you need to run the following command: mysql_secure_installation.

Thus, using the “interactive wizard”, a password will be set for access to the DBMS.

Then you need to create a user in MySQL and a database to which this user will have read / write access.

We proceed to the installation of the Hashtopolis server.

You can clone the Hashtopolis source code from the project’s git repository. We acted according to the instructions and cloned the repo, and transferred the “src” recursively to “www” from the hashtopolis directory:

1. git clone github.com/s3inlc/hashtopolis.git
2. cd hashtopolis / src
3. sudo mkdir / var / www / hashtopolis
4. sudo cp -r * / var / www / hashtopolis
5. sudo chown -R www-data: www-data / var / www / hashtopolis

Set the “KeepAliveTimeout” directive to 10 milliseconds, as the developers recommend. Justification:

In order to leverage the usage of TCP sessions in the python client, the timeout of the server needs to be set to something higher than your status update time

Our free translation: “In order to use TCP sessions in a client implemented in Python, the session timeout in the web server configuration must be set higher than the“ status update ”agent value”

Open any Apache2 configuration file (in our system, the path to file "/etc/apache2/apache2.conf") and change the value of the "KeepAliveTimeout" directive from 5 to 10.

If necessary, add alias to the Hashtopolis folder in the web server configuration (if the directory was created other than "/ var / www / html "directory). We also add the “AllowOverride All” directive to allow the use of .htaccess files in child directories.

Just in case, we added the “ServerSignature Off” and “ServerTokens Prod” directives to the “/etc/apache2/apache2.conf” configuration file, followed by restarting the apache2 service, and also indicated the allowed IP addresses for access to the “phpMyAdmin” directive “ Allow from 1.2.3.4/28 "in the config" /etc/apache2/conf-enabled/phpmyadmin.conf ". Allow access from any address to Hashtopolis or restrict locahost / PrivateIP-space - depends on the specific setup. In our case, access is allowed from any IP, as you are better off, decide this point.

It would be a good idea to restrict access to the hashtopolis directory. We went the quick and easy way - to access the "/ hashtopolis" directory, the web client must go through Basic Authentication.

Install the apache-utils package (if not done before). Of the utilities at this stage, “htpasswd" is required. Create a file and user login:

sudo htpasswd -c /etc/apache2/.htpasswd kraud

After entering the password, the user will be added to the .htpasswd file, you can check with the command "cat /etc/apache2/.htpasswd". Adding users is carried out by a similar command without the "-c" key.

Next, we configure access to the web application directory by adding directives to the apache2 / sites-available / 000-default.conf config. As a result, we got such a “directory” config:

Alias /hashtopolis "/var/www/hashtopolis"

	AuthType Basic
	AuthName "Restricted Content"
	AuthUserFile /etc/apache2/.htpasswd
	Order allow,deny
	Allow from all
	AllowOverride All
	Require valid-user 

Please note that when using “Basic-Auth”, you must additionally add parameters to the Hashtopolis agent’s configuration (file "\ hashtopolis_agent_dir \ config.json"):

{
  "url": "http://1.2.3.4/hashtopolis/api/server.php",
  "voucher": "",
  "token": "token",
  "uuid": "d54de86f-3942-4127-af63-f257c45b66cd",
  "debug": false,
  "auth-user": "kraud",
  "auth-password": "password"
}

About Hashtopolis Agent a little later.

You must also create a new database for the Hashtopolis application and the MySQL user with access to this database. Then someone who likes it more - in the PMA or in the console to create a database - this should not be difficult. We save the database name and credentials in notes - they will be needed later.

As a result of these steps, the Hashtopolis web application installation page should load at the following URL: http: /// hashtopolis:



If this does not happen, there is probably a problem with permissions on the directory "/ var / www / hashtopolis" or in the configuration of the web server (for example, an error in "Alias"). It is worth checking the URL http: ///hashtopolis/src/index.php or http: ///hashtopolis/src/install/index.php

Then, the DB Connection form will be loaded on which you will need to fill in the database connection parameters, the database name (for example 'hashtopolis') and user credentials to access it. If the connection is correctly established, the following form will open with a single “Continue” button - click it. If everything is OK, the following “Create Admin User” form will open - here we indicate the credentials of the administrator of the Hashtopolis web application:



After clicking the “Create” button, if the form was processed correctly, a document will be displayed confirming the successful installation and recommendations:

• Make sure the permissions on the "./install" directory are correct or delete this directory
• Check the contents of the file "./install/.htaccess" (should be "Order deny, allow \ nDeny from all")
• At the request directed to "./install" the status code 403 should return

After all the steps done at the URL "http: /// hashtopolis / "the Hashtopolis authentication form should be loaded:



Next (optional) configure the use of HTTPS, the connection of the security certificate - we will omit this step. We proceed to configure the Hashtopolis agent.

Agent Installation

The agents are directly involved in the “hash cracking” task - instances of 'hashcat' or another cracker are launched on the host agents. The dialogue between the Hashtopolis server and the agents occurs over HTTP / HTTPs.

The agent can be installed on systems running Windows / Linux / MacOS X. In the current version of Hashtopolis, only the Python agent is supported.

The .Net version of the Hashtopolis agent is no longer supported

. We already had a Windows 10 host that we used to “crack hashes” using 'hashcat'. Therefore, we decided not to change anything, but to take a ready-made working system for connecting to Hashtopolis.

On a Windows system, you must first install the Python 3 interpreter. Do not forget to install the “Add Python 3.x to PATH” birdie (example below):



Then, you need to download the hastopolis.zip archive from the Hashtopolis server, or otherwise deliver the archive file to the agent host.

To download agent files from the server, click on the "New Agent" from the "Agents" drop-down menu. Then we get to the section "Add new agent" where the download button will be contained - "Download".



Also, in the “Add new agent” section, the URL for connecting the agent and the button for creating a voucher will be indicated (you need to click “Create”).

In our case, the archive was saved in the directory “C: \ hashtopolis”, which will be the working directory of the agent.

For the convenience of working with the agent (viewing the log, copy / paste, etc.) we use a PowerShell session. The agent is launched by the command:

> python.exe .\hashtopolis.zip

The agent is started in debug mode using the -d option.

> python3.exe .\hashtopolis.zip -d

You can disable debugging by editing the config.json config line with the debug line: false. It’s convenient to

call help on additional arguments for running the agent command by using the “-h” option:

Hashtopolis agent commands

> python.exe. \ hashtopolis.zip -h
usage: python3 hashtopolis.zip [-h] [--de-register] [--version]
[--number-only] [--disable-update] [- debug]
[--voucher VOUCHER] [--url URL]

Hashtopolis Client v0.4.0

optional arguments:
-h, --help show this help message and exit
--de-register client should automatically de-register from server now
- version show version information
--number-only when using --version show only the number
--disable-update disable retrieving auto-updates of the client from the
server
--debug, -d enforce debugging output
--voucher VOUCHER voucher to use to automatically register
--url URL URL to Hashtopolis client API

After initialization, the agent will ask the Hashtopolis server URL (you can copy it from the "Add new agent" section): "Please enter the url to the API of your Hashtopolis installation:"

Note: the agent script must be running in the background all the time. There is a hashtopolis-runner component that runs the agent when receiving multicast messages from the server. Runner works only on linux hosts. More details here: github.com/s3inlc/hashtopolis-runner

If the URL is correct and the agent was able to connect, then a message will be received asking for the voucher: “No token found! Please enter a voucher to register your agent: ”The

agent will issue a confirmation, if the voucher code is correct and will go to the standby mode for tasks:

Successfully registered!
Collecting agent data...
Login successful!
Hashtopolis Server version: 0.10.1 ()
Client is up-to-date!
No task available!

In the "Agents" section of the table, a new agent will be added, with system information:



Hostname , GPU / CPU and other information: In the "C: \ hashtopolis" directory, the file "config.json" with the agent configuration and the log file " client.log. " Additional parameters (for example, HTTP proxy) can be added manually, according to the instruction: github.com/s3inlc/hashtopolis-agent-python We

could not find information about log rotation or the maximum size of the log, judging by the source of the agent, all stdout from the terminal is also written to file, no restrictions or checks were found.

We recommend that you immediately go to the agent settings in the Hashtopolis web interface - just click on the host name (in the screenshot above it is “DENIGMA”). It is better to install the “Trust agent with secret data” birdie, as well as indicate the owner (“Owner”):

Cracker binaries setup

The section “Crackers / Cracker Biniaries” presents the parameters of the “cracker” executable files, by default the section contains “hashcat 5.1.0”. If you need to use a different version of hashcat for any specific tasks, click “Add Version / Binary” (list of releases: github.com/hashcat/hashcat/releases ). Fill in all the fields in the form and click “Create binary”:



Be careful when filling out the form, in particular, avoid “spaces” in the “Download URL”, specify “hashcat” in the “Binary Base Name” field - the agent script will supplement the missing parts in the name executable file. So we added an additional set of executables for version 4.1.0. You can make changes by clicking on the name of the cracker in the column “Name”.

As a result of this “setup”, when creating a new task (“Tasks / New Tasks”), it will be possible to select a hashcat version.



In the Crackers section, you can also add another type of cracker, more about working with the generic cracker here .

When starting the task in the local “crackers” directory on the agent’s host, the 7z archive specified in the “Download URL” will be unpacked into the folder with the cracker ID number.


If this version of the cracker was “installed” earlier, the archive will not be downloaded again.

Create a task to crack hashes

We set up the agent and can’t wait to check it out. In order to start “cracking hashes” you need to create a list of hashes (or “Hash list”), and then specify it in the task. To create a list in the Hashtopolis web interface, click “Lists” and select “New hashlist” from the drop-down menu. On our projects, we regularly encounter NTLM / NetNTLM hashes - we will practice on them. We will use any online service and generate NTLM hashes. Then, set the remaining parameters of the list and click "Create hashlist":



An important point- all "cracked / not cracked" hashes can be seen in the created list, i.e. all hash data is associated with the list, even tasks associated with it. Therefore, be careful when deleting hash lists. And if for some reason the application modules stop working, for example, the “notifications.php” module turned off after deleting the task to which notifications were assigned, we had to click on entries in the “NotificationSetting” table.

Next, we need to add a dictionary, since we have to break the hashes we will be the “dictionary” selection (the key '-a0' in 'hashcat'). Go to the “Files” section (the “Wordlists” tab will be active), click the “Add file” button and select a local file, then click “Upload files”. We can also use the download by URL, for example, download the dictionary from a "known" repository:



Now you can start the tasks for the agent. From the Hashtopolis main menu, select “Tasks / New task”. Next, fill out the form:

1. Enter the name of the task in the "Name" parameter
2. Select the hash list in the Haslist parameter
3. Mark the dictionary from the list of "Wordlists" on the right - the name of the dictionary is automatically inserted in the "Command line" field either: Specify the path to the dictionary in the format ".. \ path \ to \ file \% filename%"
4. In the "Command line" field, enter hashcat startup parameters, for example, "-a0"
5. Priority is relevant when there are many tasks.
6. “Task notes” is used for notes; outside of Hashtopolis they will not be available.
7. You can select a color for the task with which it will be marked in the list of tasks (menu "Task / Show tasks")

We leave the rest of the parameters by default, click "Create task". After that, a new task will appear in the list of tasks:



As can be seen from the table, the agent (s) are not assigned to the task "NTLM_test_crack". You can assign an agent either from the Agent details menu in the Assignment: parameter, or select the agent from the list in the task properties and click the Assign button:



Immediately after this, the agent starts to move - the following instructions will be sent:

• Download “cracker binaries” (by default hashcat.net/files/hashcat-5.1.0.7z ). If the files exist and the version matches, the archive will not be downloaded
• Download dictionary file (if "wordlist" of hashtopolis server is specified)
• Run a short performance test
• Start Hacking Hashes

In this case, the console will display information about the stages of preparing the GPU, the selection process, system performance in "H / s", etc.

As a result of launching the task and its execution, the status of the “hash list” will be updated. The selected passwords can be viewed by clicking on the number opposite “Cracked:” in the “Tasks” table, or in the “Hash details” section:



If an agent encounters errors during the execution of tasks, the status of this agent will be changed and you will need to manually activate it. To "activate" the agent again, you need to put a bird in front of the "Activity" parameter in the agent settings. Paused status looks like this:



To prevent the agent from stopping due to errors, you need to change the “Cracker errors” parameter to one of the “Keep agent running” options (the agent will work, but errors can either be fixed in the log or rejected):

Tail

Thanks for attention! In the next part, we’ll talk about setting up notifications in Hashtopolis.