Archive for August, 2005

August 17th, 2005

Detecting leeches with ShortStat

Sometimes I see a sudden jump in the hits through ShortStat, and I’m left to wonder whether some busy site has linked to Nomadig.com, a search engine robot has gone wild or the site has been sucked by an offline browser.

The spikes of the first kind are more than welcome. More traffic here, more potential audience to be served. The second is usually okay, but there are some search engines that nobody is really interested in and their robots are not behaving properly.

The third category drives me nuts — the offline browsers are stupid enough to suck every possible page that has been linked in. This causes problems when the same page can be reached using different URLs and the site uses these variations. The same page can be fetched several times, increasing the bandwidth usage that at the end of the day costs me money. And these suckers are fast; they can fetch several pages in a second, eating bandwidth and processing power from the rest of you.

Until today, I wasn’t able to tell what caused the spike without downloading the raw log file and then trying to analyse it by finding an IP address that showed up again and again in relatively short period. As the log file contains information about all downloads, including image, JavaScript and CSS files, the task is burdensome.

A week ago I got that a-ha moment, when I suddenly realised that all required informaiton is stored in the ShortStat database. I could fetch the data with a relatively simple SQL statement and then organise it to a suitable structure for rapid analysis and print-out.

The process is very simple:

  1. Read all requests for the past X seconds from the database, organised by the IP number and the browser string.
  2. Count the number of requests per distinct IP number and browser string pair. Store the first and the last access time of these requests.
  3. Calculate the time span between the first and the last access, and use that number to calculate average hits per second.
  4. Show the number of hits, average hits per second, time span, IP address and the browser string sorted by the number of hits to the user. There is a lower limit for the hits to keep the list relatively short.

I fancied first doing that whole stuff in an SQL statement, but the timespan calculation was impossible feat for MySQL — at least without creation huge unions.

If you are interested in this, go to my ShortStat page at www.nomadig.com/shortstat/ and click link Leeches at the top of the page to see who is leeching Nomadig.com during the past 24, 48 or 72 hours.

I can make the code available, if someone is interested in using the system.

Bland fashions

Indigo Palms is a casual clothing brand for both women and men, “Men’s and women’s denim for island living”. The smallish store is located almost in the middle of the Santana Row shopping area in San Jose.

Indigo Palms on Santana Row

The clothes are quite bland and you can get similar garments from much cheaper stores, such as H&M or GAP. Or if you would like to follow the trends, go to Diesel or Urban Outfitters instead.

Indigo Palms, 355 Santana Row, San Jose, CA, USA, +1 408 551 4643

Drive-to bookstore

Barnes & Noble bookstore in San Jose seems to be in middle of nothing, at least from an European perspective. It is located next to car dealers and other semi-industrial stores.

Barnes & Noble in San Jose

The best way to get to the store is take a car. You can also walk from Santana Row, if you are not afraid of walking more than a mile.

The bookstore is quite typical Barnes & Noble store; well stocked and comparatively inexpensive (at least with the current currency rates). My personal favourite section is photography technique that has far more books than in my local shops.

www.bn.com, 3600 Stevens Creek Boulevard, San Jose, CA, USA, +1 408 984 3495

Huge upscalish mall

If you have browsed all the boutiques on Santana Row, and still have some money left on your wallet, go over Stevens Creek Boulevard to Westfield mall and spent the rest there.

The mall has two department stores: Macy’s and Nordstrom to begin with, and between these giants are over hundred specialty store. These stores cater most parts of life, so there is no need to go anywhere else.

As usual, there is also a foodcourt — this time on the second floor. If you looking for any better meal, go back to Santana Row.

www.westfield.com, 2855 Stevens Creek Blvd., San Jose, CA, USA, +1 408 248 4450

August 16th, 2005

Fixing WP links on the front page

I’m on a long vendetta to get Nomadig URLs more search engine friendly — the aim is to replace all URL parameters with a path that has the same information.

When I launched the site, I was too lazy to make a proper link structure for the journal section. That was fixed when I installed WP 1.5 and got the rewrite rules in order.

After I finished the new URL schema for reviews a week ago, the front page requested my attention. I had changed the front page links to reviews, but for some reason most of the links to journal were in form of /journal/index.php?p=….

I was using the guid field from the WordPress database. For most of the posts, it contains a proper URL with year, month, data and the post name with dashes. But for other posts, it had still index.php?p=…

To cut the long story short, I had to add a new function to the front page PHP code to create proper links to the articles, and use that function every single time the front page has a link to journal. Now the links are brilliantly search engine friendly.

Trendy fashion and furniture

Urban Outfitters is an excellent place to buy both clothes and items for home, if you like their young and trendy style.

The ground floor has big selection of garments for women and the upper floor is dedicated to men clothing and furnishing. There is also a small selection of books upstairs.

Urban Outfitters near Santana Row

The shop is especially strong in T-shirts that come in different colours and styles. Most of the shirts have some sort of graphic image or a punchline printed, so do not expect to find anything plain.

The store is not located on Santana Row, but on Olin Avenue, despite the address.

www.urbanoutfittersinc.com, 355 Santana Row, San Jose, CA, USA, +1 408 244 3329

Fashion with an edge

Diesel store on Santana Row hosts a selection of garments that have an edge or two compared to the most of the other young adults brands. You either like or hate them.

Diesel store on Santana Row

The store itself is not very huge, only one floor, and the selection is thus not wide. You can still find a few different items in every category and the shoe section is surprisingly well stocked.

The service is good. I especially like that the clerks do not immediately surround you trying to be too helpful, and drive you nuts. There was enough personal space in store to browse around in peace, but the staff was always ready to assist you would that be needed.

www.diesel.com, 378 Santana Row, San Jose, CA, USA, +1 408 241 2355

A little bit better bookshop

Borders bookstore on Santana Row has a wide selection of both fiction and non-fiction books. Especially the arts, design, photography and architecture section is good. Literature shelves are also good, but tilted towards American literature.

Borders on Santana Row

The upper floor has music collections that are okay, if you are listening to mainstream or classic music. There is also a cafe, which is a good place to rest for a while and look at the people on Santana Row — the location of the store is very central.

www.borders.com, 356 Santana Row, San Jose, CA, USA, +1 408 241 9100