This is my first project on cybersecurity and consists of a Splunk setup, connected with a Testmachine (containerized), attacked with Hydra on my Kali machine to simulate a brute-force attack.

Preparation:

• Create a container for splunk instance (.20)
  
• Create a container to test splunk forwarder, ssh enabled (.21)
  
• Configure splunk forwarder and splunk receiver
• Create simple dashboard to track ssh logins
• Test: login to testcontainer with ssh + check dashboard
• Simulate brute-force attack with hydra (.40)
  
• Configure passwordless ssh and check impact
  

Setup:

1. Create a container for splunk instance

To create the Splunk environment, I'm going to check the install notes on the server.

The container will be created on my Proxmox homelab.
To access the downloads you need to register an account on the website.

Downloads: https://www.splunk.com/en_us/download/splunk-enterprise.html?locale=en_us

Deb pkg: https://download.splunk.com/products/splunk/releases/8.2.5/linux/splunk-8.2.5-77015bc7a462-linux-2.6-amd64.deb

Demo of install process: https://www.splunk.com/en_us/resources/videos/installing-splunk-enterprise-on-linux.html

 

Before continuing I am going to create and prepare the splunkcontainer.
Checking my networklist to provide IP's. Splunk will have the .20 IP and the Testcontainer will have the .21 IP.
The reason I am using containers is because of the speed of restarts and the elasticity for adding storage/ram.

 

Specs Splunk.20:
- Ubuntu 20.04
- 20 GB (might be too little, can expand later if needed)
- 4 CPU cores
- 4GB ram
Specs Testcontainer.21:
- Ubuntu 20.04
- 8 GB
- 4 CPU cores
- 2GB ram

Starting both containers, entering the terminal to add a user.
login with root, create user benny and adding that user to the sudo group.
adduser benny

usermod -aG sudo benny

 

SSH is already activated in the containers so there's no need to install openssh-server.

Next move is to ssh into Splunk.20, update+upgrade packages and download, install + configure splunk
To download Splunk:
wget https://download.splunk.com/products/splunk/releases/8.2.5/linux/splunk-8.2.5-77015bc7a462-linux-2.6-amd64.deb
 

Now to install Splunk I need the dpkg command to unpack the .deb package (needed sudo to unpack)
sudo dpkg -i splunk-8.2.5-77015bc7a462-linux-2.6-amd64.deb

Next step is to start the Splunk installer:
splunk start --accept-license
next error: splunk: command not found

Let's find the location of splunk:
find / -type f -name "splunk" 2>/dev/null
using 2>/dev/null to get rid of all permission denied messages.

Also needed sudo to start it.

sudo /opt/splunk/bin/splunk start --accept-license

Time to configure Splunk with credentials

Login succesfull, install went pretty smoothly, I remember to encounter some issues in previous installs.
Interface looks pretty empty, before we go further, I'm going to backup the server as is.
Returning to the Splunkcontainer after boot, I found another setting to start Splunk on boot:
sudo /opt/splunk/bin/splunk enable boot-start -user splunk

 

Now to configure Splunk, we need a receiver/listener on the Splunk instance.

Use Splunk Web to configure a receiver:

1. Log into Splunk Web as a user with the admin role.
2. In Splunk Web, go to Settings > Forwarding and receiving.
3. Select "Configure receiving."
4. Verify if there are existing receiver ports open. You cannot create a duplicate receiver port. The conventional receiver port configured on indexers is port 9997.
5. Select "New Receiving Port."
6. Add a port number and save.

From <https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Enableareceiver>Following the above step gets me a listening port 9997 to use when configuring the forwarders

2. Next step: configuring forwarder in the Testcontainer

Using the universal forwarder which is also a full Splunk Enterprise instance, with more features disabled to achieve as small a resource footprint as possible.

From <https://www.google.com/search?client=firefox-b-d&q=splunk+heavy+or+light+forwarder>

Download:
wget -O splunkforwarder-8.2.5-77015bc7a462-linux-2.6-amd64.deb "https://download.splunk.com/products/universalforwarder/releases/8.2.5/linux/splunkforwarder-8.2.5-77015bc7a462-linux-2.6-amd64.deb"

Unpacking the deb-package:
sudo dpkg -i splunkforwarder-8.2.5-77015bc7a462-linux-2.6-amd64.deb

Changing to bin directory:
cd /opt/splunkforwarder/bin

Installing the forwarder, setting admin+password:
sudo ./splunk start --accept-license

Adding the forward server details:
sudo ./splunk add forward-server 192.168.1.20:9997

To send data we need to configure a monitor, because I want to track login attempts I'm monitoring the /var/log folder:
sudo ./splunk add monitor /var/log

After adding the folder monitoring, time to restart the service:
sudo ./splunk restart

Also enabling the forwarder on boot:
sudo ./splunk enable boot-start -user splunk

3. Configuring Splunk dashboard

Now the logs have been checked it is time to configure a dashboard within splunk to track the logins to the Testcontainer. For this we need a query to filter out the login attempts and success/fail logs, from this query we'll configure a dashboard to showcase the information in a more graphical way.

Dashboards are configured through search queries, so to test some queries first:
1) First I wanted to check sessions: entering "Session" in search, finding opened and closed sessions
Returning to the search with opened sessions to track abnormal amount of opened sessions, but this is also used with cronjobs.
2) Next to check for failed ssh, entering "failed" and found "failed password", looking a bit further to also check "accepted password". This will be my base query to check login attempts/successes.
3) to go deeper into the search query I am piping the results to narrow the search.
looking on the internet how to get results per host, adding the following to the query: stats count by host
"accepted password" | stats count by host
4) Renaming some servers in the graphs on the dashboard: in these results the names of a few servers are the base names (end of IP), to rename these in the graph I found the replace parameter: replace 21 with Testcontainer, 12 with Portainer
5) Also separating the attempts daily by adding another piping in the query + also sorting per day: stats count by date_mday | sort date_mday

I have several graphs now in my dashboard, ready to test a brute force attack, since the trial has not ended yet, I also configured an alert, this will save the alert in Splunk and send a mail to myself.
 

Query:
"failed password" host=12 | stats count by date_mday | sort date_mday
failed SSH attempts per host:

Query:
"failed password" | stats count by host | replace 21 with Testcontainer, 12 with portainer
Query:
"accepted password" | stats count by host | replace 21 with Testcontainer, 12 with portainer

Failed and succesful attempts to log in:

Query:
"failed password" OR "accepted password" | stats count by host

Detailed view:

4. Configuring the alert

To set an alert, I am configuring the mailsettings under server settings. Now to search for the item I want to track:
"failed password" host=21 | stats count
This counts the failed attempts to log in to the Testcontainer.

-> save as alert and configuring it to save under alerts and sending a mail.

5. Brute-force simulation

Now the fun part, Bruteforcing the ssh login on the testcontainer. Using Kali Linux for this.
- Using Hydra to execute this with the wordlist rockyou.txt

example command:
hydra -l root -P /usr/share/wordlists/metasploit/unix_passwords.txt -t 6 ssh://192.168.1.123
used command:
hydra  -l benny -P ./rockyou.txt ssh://192.168.1.21 -vvv
knowing the login name, already makes for a fast run.

After running Hydra, finding the password, the results in Splunk are visible. The alert didn't trigger due to misconfiguration, but will be fixed later on.