Having had my new PC up and running for a month now I recently treated myself to an 8GB RAM upgrade to bring me the the insane amount of 12 GB total. I did this with a plan in mind. The plan was to use some of the RAM as a RAM disk for use with VMware workstation. The idea being to create a RAM disk and install Windows XP or Linux in to the RAM disk.

Now a RAM disk isn’t all that much good because the moment you shutdown your PC or it crashes or the power fails then you will lose everything. But this wasn’t bothering me that much. What I did was setup my VM using a 6GB RAM disk for storage. Once I had it all setup and up to date I shutdown the VM and simply copied the directory over to the hard disk. Now I had my backup I fired it up once more and played around a bit.

In addition to this I added a second virtual disk on the VM which is always on the hard disk so applications and permanent data is to be saved either there or on my NAS.

So what’s it like? First off I tried it on Windows 7 and didn’t have much success. There are several free RAM disk solutions available but when I got my VM up and running I started seeing a lot of “fragmented disk” errors and after rebooting I had files missing and it wouldn’t boot fully. All the hallmarks of disk corruption or in this case RAM disk corruption. Pondering about it for a bit I figured that maybe this would only work with Error Correcting memory (ECC). I was about to give up but I decided to boot into Ubuntu and try from there.

This was a resounding success! No errors of any kind. XP installed from an ISO in less that 7 minutes using an automated install. XP boots up in approximately 8 seconds from end of BIOS to desktop and the performance is absolutely incredible. Installing Office 2007 took about 5 minutes and  clicking on Firefox has it opening in about 1 second!

What makes this all the more interesting is that VMware workstation has a feature called “Unity” which runs the VM in the back ground and provided a start menu on Gnome. This makes for a very interesting way to seamlessly integrate Windows appications into Linux.

After poking about the usual sites I settled on a bundle offered from Dabs.ie made up of the following:

CPU: AMD Phenom II 1090T Black Edition. 6 Cores at 3.2Ghz.
Motherboard: ASUS M4A87TD EVO.
RAM: 4GB DDR3
HD: Samsung Spinpoint 1TB
Case: Antec Sonata III with 500w PSU.
Video: Gigabyte nVidia GeForce GTX460 with 1GB RAM.

The total came to just a shade under €700 including shipping which was bang on my budget. The parts had all arrived last friday so after a week of having it up and running I have to say I’m very impressed with the performance of it.

This is actually my first PC that has an AMD CPU. I was initially planning to buy an Intel Core i7 but from reading various reviews the Phenom II seemed to be almost on a par with the i7 I was looking at and was a little bit cheaper too. The motherboard and the CPU support overclocking too but I haven’t gotten around to trying that out yet.

Other factors that led me to decide on this combo was the motherboard supports USB 3.0 with four slots. It also has 6 SATA connectors one of which is connected to the eSATA connector on the front of the excellent case. The case with its 500w PSU is nice and big, easy to access with plenty of room for more disks with 6 x 3.5in drive bays. My plan is to start stuffing these to capacity over the coming months. Also the case comes with a front air filter which is very handy as I’m a smoker! The PC it’s replacing is rather filthy inside to say the least. Also the case has a nice large exhaust fan which is very quiet and has a switch to allow you to choose between three speed settings.

I also opted for just 4GB of RAM in 2 x 2GB modules in keeping with a view to upgrading in the future. My primary reasoning behind this was that Windows 7 flies along nicely on just 2GB of RAM and in keeping with my budget 4GB for the moment is plenty.

As always with a new PC the immediate performance is stunning. Over the Christmas holidays I intend on putting it thorough it’s paces with a few games that I am going to treat myself to but already I’m very pleased with my purchase

First up was to try to aggregate all my online presences from Twitter, Flickr Last.FM etc. I was considering using FriendFeed but since I’ve recently been extolling the virtues of Microsoft’s Web App Installer, I noticed a new application was available to install called Amplifeeder.

As I tend to like to have quite a lot of control over my own data I decided on this over the alternatives and I have been very impressed. Unfortunately you need a Windows Server with IIS and Microsoft SQL server so if you have a *nix hosting package you will be out of luck.

So my new aggregate site is now available at http://aggregator.sweetnam.eu if you fancy a look.

The next thing on my todo list for my summer holidays is to erect an antenna on the house for yet some more long range WiFi testing. Only thing is I’m terribly afraid of heights

A total of 5368 requests, in 3 hours!

Odd things is that none of the IPs in question were typical crawlers as each IP requested an entire page at a time.

Given that there is 7 IP addresses involved and they are all in sequence I would imagine that a single Verizon customer has something nasty installed.

Here are the IPs (which I have since blocked)

65.208.151.112
65.208.151.113
65.208.151.114
65.208.151.115
65.208.151.116
65.208.151.117
65.208.151.118

The user agent:

“Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”

Grep of my logs:

robert@cache:/var/log/squid# cat access.log | grep 65.208.151 -c
5368

Whois of IP:

MCI Communications Services, Inc. d/b/a Verizon Business UUNET65 (NET-65-192-0-0-1)
65.192.0.0 – 65.223.255.255
Kintiskton LLC UU-65-208-151-112-D1 (NET-65-208-151-112-1)
65.208.151.112 – 65.208.151.119

Dodgy stuff.

I have four different networks, three wireless access points, and an unearthly amount of machines. To give you an idea of the chaos I made up a lovely diagram which is below. (There will be a fourth wireless access point in place soon but that is for a particular plan that I’m up to. More on that sometime in the future.)

The Sun Sparcs and the HP-UX machine are only powered up occasionally. I use a Dell TFT monitor which is the only display in regular use. Other than that I have a 19in Mitsubishi CRT that I use only when setting up or repairing other machines. I’ve probably mentioned this before but the room is on it’s own electrical circuit which actually has it’s own meter on it and the room is currently consuming between €1.50 and €2 of electricity per day.

I bet you thought it might be more! Most of them are desktops and spent quite a lot of time idling away consuming between 90 and 120 watts of electricity.

If you look at the diagram and are wondering about the External WLAN limited to 1Mb, this is the access point and network that I use whenever I have a nasty virus and or spyware infected PC or laptop for repair.

Anyway, Click on the diagram for a bigger version:

As it happens, while tidying up I found an old spare disk and I decided to give it a new lease of life. Again as it happens I needed another machine for a while for a particular task that this machine would be quite suited to. So now it is alive once more and is happily pushing packets across the internet.

I was quite pleased with the results so much so that I decided that it could run it’s very own web server.  There is just a single html page with it’s story about how an open source operating system (FreeBSD in this case) gave it a new lease of life.

Have a look over here.

Things must be bad when my imagination starts assigning personalities to computers!

The keen eyed among you may have noticed from the phpsysinfo page that it is on a different network to my other web servers. This is a clue as to it’s purpose but unfortunately I can’t divulge it’s purpose here yet!

Mind you, every now and then something comes along to remind you that maybe Adobe Flash is not so bad after all. And maybe, just maybe there is a use for it other than useless website intros and You*.com videos.

So here is a virtual hamster: http://abowman.com/google-modules/hamster/

But just to prove that Adobe Acrobat is still hateful, the latest version of Acrobat Reader weighs in at 33.5MB. To put that into comparison Open Office can create the damn things and it’s free! And it is only a 142MB (JRE Bundled) download but you get an entire office suite of software! Word processor, spreadsheet and you get the idea. You get more than just something for just reading documents!

As if that isn’t hateful enough just consider that Open Office is considered to be among the slowest pieces of software available – yet it loads up in about 30 seconds compared to about 3 minutes for Acrobat reader on the exact same P.C! Grrr..

But if it is just the ability to read PDF documents that you are after then FoxIt Reader is free and only weighs in at 2.57MB and takes but a split second to open.

And it is better than having to download a 200MB update for a program that was originally 33.5MB to download.

So first up was the retirement of my IIS server on which my tech blog used to reside. I have migrated that blog over to WordPress and it along with this blog and my main site have been relocated on to a new Debian 64bit server which was previously the already mentioned IIS server.

The migration also meant that I could catch up on some tweaks that I had long been putting on the back burner. As my main site is running Mediawiki I have configured the new server so that Mediawiki now uses memcached to help speed up queries and reduce hits to the database. It has had a bit of an effect already. in keeping with the desire to reduce queries on the database I installed Donncha’s excellent WP Super Cache plugin on both WordPress blogs. To give an idea of the difference that made a quick benchmark was called for.

Using Apache Bench I made requested a blog entry 100 times over 10 concurrent connections:

ab -n 100 -c 10 http://tech.sweetnam.eu/2008/06/even-intel-wont-touch-vista/

Without WP Super Cache the results were 20.72 seconds to complete with 71.47kbytes/sec

With WP Super Cache the results were 2.287 seconds to complete and 658.38kbytes/sec

However there is a caveat with those results as each test first passed through my reverse proxy but nonetheless the results are pretty conclusive. Another configuration change that I made that may skew those results is that I have disabled Apache logging on the web server. As all traffic first passes through my reverse proxy I use its logfiles for analysis. Interestingly from another test I tried recently, disabling logging on Squid Cache had absolutely no effect on its performance!

With everything now happily in place on the new server it was time to shut down and power off both of my Sun Machines. They will be ressurected for anytime I wish to play with Solaris. My mailserver remains running on my Poweredge 2800 although I took the opportunity to upgrade to the latest version of Zimbra today.

After all todays tweaking and migrating I am now left with my Smoothwall firewall performing NAT, my reverse proxy which I decided to keep up and running as it is outstanding for filtering unwanted traffic, the new web server and my existing mail server.

It’s probably still a tad excessive though! Next up is to change the theme for this site as I can’t configure the sidebars and the search option is strangely missing!

So why does it perform so well especially when compared to Apache running on its own?

The main reason is caching. When someone makes a request the proxy server checks to see if it has it locally and if it does it will serve that up directly instead of requesting it from a backend server. This of course depends on a number of factors. Primarily it will only cache static content such as images, css and javascript files as well as good old html. It will not cache dynamic content such as the contents of a blog post such as this one (although this can be done using memcached), however as it may well cache the ancilliary files associated with the page. The greater the number the times the same content is requests then the greater the chance that Squid will keep it in memoy increasing performance even more.

To use this post as an example, all the images, theme and javascript have more than likely been served to your browser by the proxy server, all that was requested from the web server itself was the text which has been pulled from a database using Apache and PHP.

So how much better does squid serve static content from its cache compared to Apache?

The quick answer is that it serves it staggeringly faster than Apache. For my test I used a single jpeg image that was 73KB in size. I then set up two virtual servers on my webserver and copied the image into both. I configured my internal DNS so that one of the sites would be served direct by the webserver and the other would be accessed via my reverse proxy.

Benchmarking was performed from a my main workstation using Apache Bench (ab). I used a concurrency of 10 and requested the image 100 times from each test site. So first up is the direct request:

Concurrency Level:      10
Time taken for tests:   2.621 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      7510044 bytes
HTML transferred:       7465900 bytes
Requests per second:    38.16 [#/sec] (mean)
Time per request:       262.082 [ms] (mean)
Time per request:       26.208 [ms] (mean, across all concurrent requests)
Transfer rate:          2798.37 [Kbytes/sec] received

That’s not too bad but here are the results when served by the reverse proxy:

Concurrency Level:      10
Time taken for tests:   0.640 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      7510100 bytes
HTML transferred:       7465900 bytes
Requests per second:    156.35 [#/sec] (mean)
Time per request:       63.960 [ms] (mean)
Time per request:       6.396 [ms] (mean, across all concurrent requests)
Transfer rate:          11466.60 [Kbytes/sec] received

It performed twice as fast when passed through the proxy server which is phenomenal!

However some of this performance could be attributed to differences in hardware. The webserver is a Sun 900Mhz UltraSPARC III and the reverse proxy a 3Ghz P4 but nonetheless it does demonstrate that the reverse proxy greatly accelerates performance when serving static content.

As all of you know, running a blog or any site can have it’s problems and almost all these problems are dealing with deluges of spam and trying to block unwanted crawlers siphoning information from your site. There are many excellent methods to reduce this unwanted traffic namely Akismet or Fail2ban however given the nature of my setup here Akismet is not as effective as it should be and fail2ban is not an option but this is where Squid comes into its own.

Squid is utterly configurable with so many options that you could find yourself easily overwhelmed. But there is one particular feature that is absolutely stellar in allowing me to control who and what accesses my sites. That feature is using Squid’s built in ability to use regular expressions to deny access based on a visitor or bots browser string. Apache can perform similar functionality using mod_rewrite but I find that having Squid to do the dirty work is a much more elegant solution as it flately denies access to the backend servers in the first place.

So how does one go about doing this? Well you can pop on over to my Wiki where I have put together a HowTo which also has the regular expressions that are used to block all the bad browsers.

Needless to say my setup is vastly different than most but nontheless my HowTo could possibly help others in tweaking their methods of keeping the bad guys at bay.

If anyone finds it useful or spots something that could be improved I would be happy to hear from you.