First off I better explain what a WISP , it means Wireless Internet Service Provider. For this sorry tale the WISP in question is OceanTelecom who operate in West Waterford/East Cork.

I got a call the other day from my friend’s wife who was having problems receiving mail from her eircom.net account when using POP to retreive it. Sometimes she would get some mail and other times none. Using the webmail interface to her account she could see her mail there. All this started she claimed around two weeks ago.

Strange thought I, but then she mentioned that she had gotten a few spam reports in her mail.

Even stranger thought I, so I asked her to forward one of them on to me and here’s what I saw:

This mail was generated automatically from Endian Firewall, which runs on
efw1-oceantelecom.localdomain.(none) for scanning all mails for spam and viruses.

In a mail sent to you a virus has been found.

Virus name: Suspect.Bredozip-zippwd-2
Sender of the email:  “DHL Manager Felipe Dove” <shipping@dhl.com>
Subject: DHL delivery problem number 25130.
Connection date: POP3 from 149.5.34.3:11778 to 159.134.198.135:110
Message File: Per instruction, the message has been deleted.

Instead of the infected email this message has been sent to you.

Regardless of the fact that it was a virus, it was the very fact that it had been intercepted before it got to her PC was annoying her. Her POP session to eircom’s mail server was intercepted by her ISP. Her ISP acted as a POP proxy without her permisison. Therefore her ISP is effectively snooping on her mail. You can see clear as day in the report above that their firewall intercepted her connection to eircom’s POP3 server.

As you can imagine this really, really annoyed her so she rang ComReg (Ireland’s Communications Regulator) to find out if OceanTelecom were allowed to do this. ComReg said it was a grey area and advised her to contact the office of the Data Protection Commissioner which she duly did. The advice she received from the DPC was that they should not be snooping on her mail as her mail is not being hosted by her ISP.

With this information in hand, she rang OceanTelecom to compain and promptly received torrents of abuse from the owner! Ranting and raving about how he is protecting his network, etc, etc and if she didn’t like it she could cancel her account! How about that for customer service?

But when she informed him that she had already contacted ComReg and the DPC he terminated the call! He hung up a call from a loyal customer of over two years!

She rang me yesterday to tell me what had happened and that that she was naturally going to change her ISP which is only right IMHO.

So, for anyone looking to choose an ISP keep this information in mind. An ISP is an internet service provider. Their only obligation should at it’s most basic level to provide you with access to the internet and nothing else. Everything else should be optional. How you use your internet connection should be of no interest to your ISP once you keep within the terms of your contract, the laws of the land and adhere to their fair use policy. You can view eircom’s policy here.  Some choice quotes from their policy are:

eircom net will use its reasonable endeavours to prevent unauthorised access to the Service by third parties, but shall have no liability to the Customer for any unauthorised access to the Customer’s computer system. The Customer is responsible for selecting and properly using any security procedures made available by eircom net as well as other procedures and measures necessary to safeguard and back-up the Customer’s files, data and programs or any other form of information

and

You acknowledge that eircom net has no control over the information which can be accessed by using eircom net services and that we do not examine the use to which you or other users put the Services or the nature of the information you or they are sending or uploading. We therefore exclude all liability of any kind for the transmission or reception or such information of whatever nature.

Pretty much common sense. The onus is on the customer to remain secure, eircom as an ISP only provide a service which is more than can be said for OceanTelecom.

However I understand that OceanTelecom is a privately owned business and ultimately it is their network and they can pretty much do what they want but I would not have expected that to include interfering with clients  e-mail that is hosted elsewhere. I certainly wouldn’t like my ISP reading my mail before me. Some serious privacy implications there.

If you are an OceanTelecom customer or are considering becoming one, based on the above I would suggest you avoid them or terminate your subscription. If anything the shocking and abusive customer service alone should be good enough reason.

A total of 5368 requests, in 3 hours!

Odd things is that none of the IPs in question were typical crawlers as each IP requested an entire page at a time.

Given that there is 7 IP addresses involved and they are all in sequence I would imagine that a single Verizon customer has something nasty installed.

Here are the IPs (which I have since blocked)

65.208.151.112
65.208.151.113
65.208.151.114
65.208.151.115
65.208.151.116
65.208.151.117
65.208.151.118

The user agent:

“Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”

Grep of my logs:

robert@cache:/var/log/squid# cat access.log | grep 65.208.151 -c
5368

Whois of IP:

MCI Communications Services, Inc. d/b/a Verizon Business UUNET65 (NET-65-192-0-0-1)
65.192.0.0 – 65.223.255.255
Kintiskton LLC UU-65-208-151-112-D1 (NET-65-208-151-112-1)
65.208.151.112 – 65.208.151.119

Dodgy stuff.

Strange.

“I know my DNS server was down for maintenence but I definitely recreated my DNS Zone.”

Quick check,

“Bugger! I forgot to add an entry for blog.sweetnam.eu!”

Oh well, I’m back now.

I have four different networks, three wireless access points, and an unearthly amount of machines. To give you an idea of the chaos I made up a lovely diagram which is below. (There will be a fourth wireless access point in place soon but that is for a particular plan that I’m up to. More on that sometime in the future.)

The Sun Sparcs and the HP-UX machine are only powered up occasionally. I use a Dell TFT monitor which is the only display in regular use. Other than that I have a 19in Mitsubishi CRT that I use only when setting up or repairing other machines. I’ve probably mentioned this before but the room is on it’s own electrical circuit which actually has it’s own meter on it and the room is currently consuming between €1.50 and €2 of electricity per day.

I bet you thought it might be more! Most of them are desktops and spent quite a lot of time idling away consuming between 90 and 120 watts of electricity.

If you look at the diagram and are wondering about the External WLAN limited to 1Mb, this is the access point and network that I use whenever I have a nasty virus and or spyware infected PC or laptop for repair.

Anyway, Click on the diagram for a bigger version:

As it happens, while tidying up I found an old spare disk and I decided to give it a new lease of life. Again as it happens I needed another machine for a while for a particular task that this machine would be quite suited to. So now it is alive once more and is happily pushing packets across the internet.

I was quite pleased with the results so much so that I decided that it could run it’s very own web server.  There is just a single html page with it’s story about how an open source operating system (FreeBSD in this case) gave it a new lease of life.

Have a look over here.

Things must be bad when my imagination starts assigning personalities to computers!

The keen eyed among you may have noticed from the phpsysinfo page that it is on a different network to my other web servers. This is a clue as to it’s purpose but unfortunately I can’t divulge it’s purpose here yet!

So first up was the retirement of my IIS server on which my tech blog used to reside. I have migrated that blog over to WordPress and it along with this blog and my main site have been relocated on to a new Debian 64bit server which was previously the already mentioned IIS server.

The migration also meant that I could catch up on some tweaks that I had long been putting on the back burner. As my main site is running Mediawiki I have configured the new server so that Mediawiki now uses memcached to help speed up queries and reduce hits to the database. It has had a bit of an effect already. in keeping with the desire to reduce queries on the database I installed Donncha’s excellent WP Super Cache plugin on both WordPress blogs. To give an idea of the difference that made a quick benchmark was called for.

Using Apache Bench I made requested a blog entry 100 times over 10 concurrent connections:

ab -n 100 -c 10 http://tech.sweetnam.eu/2008/06/even-intel-wont-touch-vista/

Without WP Super Cache the results were 20.72 seconds to complete with 71.47kbytes/sec

With WP Super Cache the results were 2.287 seconds to complete and 658.38kbytes/sec

However there is a caveat with those results as each test first passed through my reverse proxy but nonetheless the results are pretty conclusive. Another configuration change that I made that may skew those results is that I have disabled Apache logging on the web server. As all traffic first passes through my reverse proxy I use its logfiles for analysis. Interestingly from another test I tried recently, disabling logging on Squid Cache had absolutely no effect on its performance!

With everything now happily in place on the new server it was time to shut down and power off both of my Sun Machines. They will be ressurected for anytime I wish to play with Solaris. My mailserver remains running on my Poweredge 2800 although I took the opportunity to upgrade to the latest version of Zimbra today.

After all todays tweaking and migrating I am now left with my Smoothwall firewall performing NAT, my reverse proxy which I decided to keep up and running as it is outstanding for filtering unwanted traffic, the new web server and my existing mail server.

It’s probably still a tad excessive though! Next up is to change the theme for this site as I can’t configure the sidebars and the search option is strangely missing!

So why does it perform so well especially when compared to Apache running on its own?

The main reason is caching. When someone makes a request the proxy server checks to see if it has it locally and if it does it will serve that up directly instead of requesting it from a backend server. This of course depends on a number of factors. Primarily it will only cache static content such as images, css and javascript files as well as good old html. It will not cache dynamic content such as the contents of a blog post such as this one (although this can be done using memcached), however as it may well cache the ancilliary files associated with the page. The greater the number the times the same content is requests then the greater the chance that Squid will keep it in memoy increasing performance even more.

To use this post as an example, all the images, theme and javascript have more than likely been served to your browser by the proxy server, all that was requested from the web server itself was the text which has been pulled from a database using Apache and PHP.

So how much better does squid serve static content from its cache compared to Apache?

The quick answer is that it serves it staggeringly faster than Apache. For my test I used a single jpeg image that was 73KB in size. I then set up two virtual servers on my webserver and copied the image into both. I configured my internal DNS so that one of the sites would be served direct by the webserver and the other would be accessed via my reverse proxy.

Benchmarking was performed from a my main workstation using Apache Bench (ab). I used a concurrency of 10 and requested the image 100 times from each test site. So first up is the direct request:

Concurrency Level:      10
Time taken for tests:   2.621 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      7510044 bytes
HTML transferred:       7465900 bytes
Requests per second:    38.16 [#/sec] (mean)
Time per request:       262.082 [ms] (mean)
Time per request:       26.208 [ms] (mean, across all concurrent requests)
Transfer rate:          2798.37 [Kbytes/sec] received

That’s not too bad but here are the results when served by the reverse proxy:

Concurrency Level:      10
Time taken for tests:   0.640 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      7510100 bytes
HTML transferred:       7465900 bytes
Requests per second:    156.35 [#/sec] (mean)
Time per request:       63.960 [ms] (mean)
Time per request:       6.396 [ms] (mean, across all concurrent requests)
Transfer rate:          11466.60 [Kbytes/sec] received

It performed twice as fast when passed through the proxy server which is phenomenal!

However some of this performance could be attributed to differences in hardware. The webserver is a Sun 900Mhz UltraSPARC III and the reverse proxy a 3Ghz P4 but nonetheless it does demonstrate that the reverse proxy greatly accelerates performance when serving static content.

I thought I was supposed to have been upgraded to 10Mb as I previously was on 4Mb so I guess that’s one for Monday morning. My upload is now 672k so that should speed things up a bit for those of you who visit.

First things first – The details:

• The can comprises of a Tia Maria Lusso presentational can which measures 110mm in diameter and is 260mm long. From various sites on the web about making cantennas I figured the best position for the USB adapter to be mounted would be about 40mm from the base of the can and mounted in such a way that if you hold the can horizontal and look into it the WiFi adapter would be vertical.
• My wireless adapter is a NetGear WG-111.
• And as I don’t have a Windows laptop I was using an XP virtual machine running on VMware Fusion on my MacBook.

Today then I decided to pop over the Ferry Point which is the Waterford side of the Blackwater Estuary. It’s about a 6 mile drive to get there but when you do you are somewhere about 750 yards to 1 mile straight across the river from Youghal town.

The handy thing about using a Virtual machine was that I could use the MacBooks own Wifi to see what is around locally and compare it to the cantenna connection in the virtual machine. As a result the MacBook picked up two access points which I presume were the two houses within 200 yards of me.

When I connected the NetGear  USB adapter without the cantenna it picked up the same two access points however when I connected up the Netgear in the cantenna and pointed it towards Youghal then I was in for a big surprise! Have a look at this:

First results with cantenna

First results with cantenna continued

A total of 18 and that was just pointing to the middle of the town! Panning around found quite a few more but not my own however as my house is on street level parallel to two other streets before the riverbank. What is surprising is the strength of the signals.

While on my way home I stopped at the Waterford side of the old Youghal Bridge and ran some more tests. This location which you will be able to see from one of the pictures below is about two miles straight to the Town Hall in the town centre. I was amazed that once again I was able to see som many connections including an open one which I know exactly where it is located. I can’t give you the exact location but I can tell you that it is near the town centre and I was able to make a connection, get an IP address and browse a few web sites.

Now for some more pictures:

View from the Old bridge

View across to Youghal

Bad exposure of my cantenna

Windows VM on my MacBookView across to Youghal

This is bad because it seriously skews my statistics for one and very bad because it is chewing up a lot of my available bandwidth. So what is one to do?

As I noted previously, AVG uses a few unique user agent strings. These can be used in a .htaccess file to deny access to Linkscanner or as in my case redirect the request to a certain page. Unfortunately my ability with using regular expressions is pretty limited to say the least but while browsing a discussion about Linkscanner on reddit.com last night someone posted a solution. That someone was none other than Pádraig Brady who is a frequent contributor to the Irish Linux Users Group.

Rather than post Pádraig’s solution here I will link to it instead. However instead of directing AVG users back to the AVG site as Pádraig’s example does – I redirect them to a custom page I made earlier. I was going to link to my custom page but since I will be using it to keep a tally of AVG hits I decided not to link to it here.

To get an idea of how much extra traffic is generated as a result of Linkscanner consider that this site is relatively light on traffic but in the two hours since I started redirecting AVG users, the page mentioned above has been hit over 240 times!

Bad AVG, very bad.