GoatCounter 1.2: unique visits, events, password auth, and more

GoatCounter 1.2 is out. Some highlights:

• Unique visit tracking

  GoatCounter now tracks “visits” in addition to pageviews. This is probably the most-requested feature.

  The way this works is that a hash of the IP address, User-Agent, siteID, and a salt (which changes every 8 hours) is generated for every pageview and used to link up individual pageviews during this 8-hours. After that, everything is “forgotten” and subsequent visits are not connected.

  There’s a longer doc on how this works and the rationale at docs/sessions.markdown.

• Password authentication

  I thought that email-based authentication would be simpler and more user-friendly, but I was wrong about that (some context). This version supports password authentication, and the email auth will be removed in the next version as there’s little point to keep maintaining it, and removing it will reduce the maintenance burden.

• Bot detection

  A lot of web traffic is generated by bots, but you rarely want this to show up in your analytics stats.

  Blocking 100% of bots that go out of their way to mark as regular browsers is almost impossible without invasive profiling, but the current solution seems to work better than most other (public) ones I could find. Google Analytics probably still does a better job at this, as they can link up data from reCAPTCHA and whatnot, but not doing this is probably considered an advantage by most users ;-)

  I spent a long time looking at existing solutions and the GoatCounter logs; it’s one of those unsexy things that doesn’t really show in the UI 😅 There’s probably still a bit that can be improved here; it’s an on-going process.

  The pageviews are still stored in the database, just marked as “bot” and not shown anywhere except in the export. I might make an option for showing them in the future.

• Events

  There’s now better support to track events, such as people following an external link or clicking a button. The count.js script automatically binds to elements with the data-goatcounter-click attribute set, and they’re marked as “event” in the backend. See the “site code” docs in the backend for details.

• Better support for campaigns

  The tracking script now sends the query parameters separately so it’s easier to track campaign parameters from the URL, such as utm_campaign and ref. If such a parameter is present it will override the referrer from the HTTP header.

  This also means people should see fewer duplicates such as /path and /path?utm_source=some-random-newsletter-that-linked-your-site.

  Still some work to be done to give some better insight in this, such as separating out “top referrers” and “campaigns”.

• WordPress plugin

  There’s now a WordPress plugin; turned out that adding GoatCounter to WordPress isn’t as easy as just “paste site code in some setting field” as WordPress doesn’t really offer such a field.

  It’s pretty basic (I’m not a WordPress dev), and I’ll slowly continue adding some more options to make several common customisations easier.

• Many small improvements and bug fixes

  It’s almost 2 months of work, and there have been many small changes, fixes, and improvements. I didn’t keep track of them all 😅

As always these changes have been live on goatcounter.com as they’re being developed in CI fashion.

What’s next?

I’d like to make more frequent smaller releases; 1.2 has a few big changes that impact quite a lot. One reason I waited a bit with the 1.2 release is so that I can have a “big” release for “advertising” purposes 😎 Right now the only advertising I’ve done was a Lobsters and HN thread in January, which gave me a surprising amount traffic, but now is probably the time to step up the advertisement/marketing thing a wee bit. Having a product is great and all, but if no one has ever heard of it it’s kinda useless (also see financials below).

Some specific things on the short-term agenda:

• I’m really pushing the limits of the flexbox-based bar charts; showing both unique visits and pageviews in the same chart is a bit clunky (although it does work), and having thousands of DOM elements can actually be quite slow.

  I did a review of various JavaScript charting libraries, and decided to go with d3, since this gives me a lot of power without having to include a megabyte of JavaScript.

  This will also give me the opportunity to improve the a11y of the charts, since they’re not very usable with a screen reader right now.

• Add OS stats, as well as improve the User-Agent parsing. The library I use for this now is pretty suboptimal, and there doesn’t seem to be a better one for Go right now. I think I’ll have to write my own one (update since I wrote this last week: I already made a library and a PR for this).

• Send daily/weekly/monthly email reports, so you don’t have to log in every day to see how your site is doing. Shouldn’t be too hard to add, and quite a few people have requested this.

• Start thinking about adding “goals” and “funnels”; now that we track visits, we can have some insight in this. Probably not too hard technically, just need to think about a good UI.

Status of goatcounter.com, financials

GoatCounter has about 600 active sites – where “active” means “received at last one pageview in the last 3 days” – and 1,000 sites in total. There are about 75,000 pageviews/day on average, with the biggest spike being 745,033 in a day.

I’ve been meaning to create a maintained “financials” page, but haven’t gotten around to it yet. A quick overview of the financials as of May 10th:

Income

  Personal plans      23, € 44
  Personal plus plans  4, € 20
  Business plans       3, € 45
  GitHub Sponsors     15, €142 (×2 from pledged amount, since GitHub doubles it)
  One time donations   2, € 58
  Patreon             16, € 24 (platform costs already deducted)

Expenses

  Stripe costs           -€ 14
  Linode VPS             -€ 22
  Heroku                 -€  7 (Chat widget; easier than running NodeJS myself)

Total revenue: €290

Not too bad considering it’s fairly new and I did almost no marketing at all, but to make it truly sustainable roughly €2,000/month (before tax) is needed, so still some way to go 😅

There are about 61 people who contribute financially, so it’s a subscriber rate of about 9%. I’m not entirely sure what an average free/subscriber ratio is for these kind of things, but 9% seems fairly good to me.

One piece of rather good news is that I applied for a grant from NLnet’s NGI0 “Privacy Enhancing Technologies” fund and got approved, giving me €25,000 over the next year or so. This is pretty critical because without it I’d have to find a regular day-job sooner rather than later since I haven’t had any serious income since February last year, and my bank account is not exactly at an all-time high 😬

This will allow me to work on GoatCounter for the next year if I’m careful enough with my money, by which time it should hopefully be profitable enough to be sustainable.

All in all, things seem to be going in the right direction, somewhat to my own surprise 😅