Directory Data

If you haven’t guessed it already, we love data in open formats. Good quality, easily accessible data makes our lives easier, and causes children across the nation to beam with joy at the idea that they won’t have to copy a table from a Word document buried in a Zip file attached to an email.

In a continued drive to making all our data 5-star quality, I’m pleased to announce that we’ve made a few improvements to our Staff Directory beta. In addition to getting hold of people’s profiles in HTML using your browser (for example, see mine) you can now request them in three other delicious formats: JSON, RDF/XML and vCard.

The first two, JSON and RDF/XML will make the developers amongst you über happy. You can request them either by slinging appropriate http-accept headers to the usual URI for a person (http://lncn.eu/me/{account_name} is the canonical one), where application/json or application/rdf+xml will get you what you desire. Alternatively, you can hit up http://lncn.eu/me/{account_name}.json or http://lncn.eu/me/{account_name}.xml for the same thing.

The vCard format is of more interest to all, and provides a stupidly easy way to get a person’s details into your address book. By visiting http://lncn.eu/me/{account_name}.vcf (or by clicking the link at the bottom of any Directory profile) you’ll be given that person’s vCard, presenting their name, job title and contact details all in the industry standard machine readable format. It’s literally a matter of one or two clicks (or taps) to get information from the Directory into your computer’s (or phone’s) address book. If you want, you can download mine to see what I mean.

Continue reading “Directory Data”

All About You

As you’ll know if you follow the adventures of Alex and myself we’ve been playing around with our new-look staff directory (give the beta a whirl). We’ve rebuilt it from the ground up to be stupidly fast, using bleeding-edge search technology all wrapped in Web 2.0 goodness to deliver your search results as quickly as possible. After all, why would you want to hang around waiting for half a second when we can have the number you’re looking for in a quarter of that time?

Directory search is awesome, but we thought we could do more with this information. We could take a person’s staff directory entry and make it a little bit more epic, as well as a bit more useful on the internet as a whole. So we did, and we’re happy to introduce the (beta) of staff profile pages – for examples see mine, Alex’s, Joss’s and Paul’s.

Continue reading “All About You”

Why RFID? Why Not?

I’ve recently had a small ponder on the subject of the University’s ID cards and ways to make them more useful, since user experience is something I care strongly about and how we identify people is a big part of that.

Something I’d really like to see is the University have a proper unified access system, after recently talking to a student who carries their ID card and three other completely blank white cards to allow them access to various rooms and buildings. One other staff member sports two white cards (completely blank) and a keyfob on top of their ID card. At one point I carried my student ID card, staff ID card and no fewer than four mysterious blank white cards all issued by various University departments.

Asking someone to carry multiple additional credit card sized pieces of plastic and potentially clip other things to their keyring is over the top when people are already carting around driving licences, debit cards, store loyalty cards and lord knows what else. On top of the sheer quantity of plastic is the inherent flaw in having blank cards – yes they provide no information on which doors they open which is great if they’re lost, but equally they provide no information on which doors they open which is really annoying if you have more than one of them.

I propose that to solve this problem the University replaces all staff ID cards with RFID enabled ones (Mifare 1K cards for preference, the exact same ones as the current mysterious blank white cards). As soon as this is done they issue all new students and card replacements with RFID cards.

A blank white Mifare 1k card, bought in a pack of 100, will cost around 70p. Yes this is more than the current blank cardstock, but the extra expenditure is virtually nothing in the grand scheme of things (you could easily spend 50-60p on giveaway pens). The card externally looks and feels exactly the same, so all of the University’s current information and security features will work, including the barcode to retain perfect backwards compatibility. In short there is no risk whatsoever to existing ID card processes and systems in moving from ‘dumb’ cardstock to RFID cardstock.

Since the new cards follow the exact same standards as the current mysterious blank white cards a person’s ID card can now be treated as a security card. There is absolutely no requirement at all for multiple cards, since in the past I’ve had my blank white cards merged into one. Keep a note of which person each ID card belongs to and should they lose a card or leave you simply look up their name in the system and revoke the card access, then as soon as they get a new ID card you restore their access.

In theory such as system (should our building security sport half-decent interfaces to anything) could even be tied into HR and student information systems, which is where what I do comes in. Imagine that when a student enrolls their card is printed as usual and automatically programmed to let them into the rooms which are appropriate for their course. When a member of staff is issued with their ID card there’s no subsequent waiting for a departmental administrator to fish a mysterious blank white card from a filing cabinet somewhere, they’re simply instantly granted access to their building and office. Finally, when someone moves or leaves their security access is seamlessly updated.

Yes, RFID will cost more (pennies per card), but as far as I can see all we’re going to do is maintain compatibility with barcode based systems, streamline and simplify the building access systems, and since Mifare 1k is a fairly de-facto standard for RFID applications (cashless systems, security, you name it) then we’ve put in part a major part of infrastructure for future developments.

So what do you think? Arguments for and against are more than welcome.

Unlocking Gateway

My most recent project (following on from Jerome, slotting in around the rest of the Summer’s “oh God, students are coming back, fix everything” mayhem has been to look at Gateway, more specifically to take Gateway and give it some extra awesome based around exploratory work we did with MPath. Here’s a quick breakdown of what you can expect when it’s turned on.

Very pretty. Very fast.

We’ve moved from a CWD 2.3 based design to our brand new CWD 3.0. This gives us a huge number of improvements in just about every area; layout, typography, readability, accessibility, compatibility, mobile readiness, new JavaScript frameworks, massively improved speed optimisations and more. We’ve compressed files and shaved off unnecessary bytes from almost the entire framework, making it load astonishingly quickly even over mobiles.

CWD 3.0 is also served over a blazingly fast content delivery network. Specifically we’re using Rackspace Cloud Files, who pipe their content to end users over the Akamai network. Put simply, this means that content such as the styling and images is delivered to your browser from a point much closer in internet space, regardless of where you are. If you want to access the Gateway from Bhutan then instead of serving all the content from a box in Lincoln some of it will come from whichever one of Akamai’s 84,000 servers happens to be closest. The result is a blisteringly fast experience, and since a lot of the Akamai servers are hooked straight into providers’ networks then it’ll still be quick even on mobile devices.

As mobile as your mobile.

We’ve made sure that Gateway optimises itself on the fly for most modern mobiles, and since it uses the CWD for its underlying design it’ll instantly take advantage of future improvements as we deploy them. We’ve sat down with our desks full of phones and tablets and tested things to make sure they’re easily read and are simple enough to use with just one finger.

Smarter.

Gateway now runs on a brand new system, meaning we can give it some extra smarts. If you visit Gateway and we’ve noticed there’s a problem with Blackboard then you’ll be told about it, meaning less clicking a link and waiting whilst it doesn’t load. It can tell you the local weather forecast, show you which trains are running late and even give you notices specific to your location, all in one place.

It’s all about you.

Sign in to the Gateway or use any of the services using Single Sign-In and it’ll gather all kinds of information you might find useful and display it for you. Your next lecture, assessment deadlines, how many library books you’ve got out and more is right at your fingertips.

Rock solid.

Gateway has moved from one very resilient platform to another even more resilient platform. Located off-campus on a world-class hosting platform Gateway can survive snow, flood and even builders cutting through power lines to provide you with updates even if everything else is going wrong.

Useful numbers on our wall!

After a lengthy run-up, our Panic-inspired status boards are partially here! We have two Samsung 460MX-3 displays (one for each of our two offices), kitted out with the Network module which gives them an embedded system we can control remotely.

We’re currently viewing a custom web page – one for each office – displaying some key stats from our ticketing system. This is only a first step though, we’ll be developing the status boards to tell people what they need to know, be that how servers are performing, how many tickets they’ve got open or how much traffic is flowing through a given website.

And now, photos!

The Re-Architecting of Jerome

Over the past few days I’ve been doing some serious brain work about Jerome and how we best build our API layer to make it simultaneously awesomely cool and insanely fast whilst maintaining flexibility and clarity. Here’s the outcome.

To start with, we’re merging a wide variety of individual tables ((Strictly speaking Mongo calls them Collections, but I’ll stick with tables for clarity)) – one for each type of resource offered – into a single table which handles multiple resource types. We’ve opted to use all the fields in the RIS format as our ‘basic information’ fields, although obviously each individual resource type can extend this with their own data if necessary. This has a few benefits; first of all we can interface with our data easier than before without needing to write type-specific code which translates things back to our standardised search set. As a byproduct of this we can optimise our search algorithms even further, making it far more accurate and following generally accepted algorithms for this sort of thing. Of course, you’ll still be able to fine-tune how we search in the Mixing Deck.

To make this even easier to interface with from an admin side, we’ll be strapping some APIs (hooray!) on to this which support the addition, modification and removal of resources programmatically. What this means is that potentially anybody who has a resource collection they want to expose through Jerome can do, they just need to make sure their collection is registered to prevent people flooding it with nonsense that isn’t ‘approved’ as a resource. Things like the DIVERSE research project can now not only pull Jerome resource data into their interface, but also push into our discovery tool and harness Jerome’s recommendation tools. Which brings me neatly on to the next point.

Recommendation is something we want to get absolutely right in Jerome. The amount of information out there is simply staggering. Jerome already handles nearly 300,000 individual items and we want to expand that to way more by using data from more sources such as journal table of contents. Finding what you’re actually after in this can be like the proverbial needle in a haystack, and straight search can only find so much. To explore a subject further we need some form of recommendation and ‘similar item engine. What we’re using is an approach with a variety of angles.

At a basic level Jerome runs term extraction on any available textual content to gather a set of terms which describe the content, very similar to what you’ll know as tags. These are generated automatically from titles, synopses, abstracts and any available full text. We can then use the intersection of terms across multiple works to find and rank similar items based on how many of these terms are shared. This gives us a very simple “items like this” set of results for any item, with the advantage that it’ll work across all our collections. In other words, we can find useful journal articles based on a book, or suggest a paper in the repository which is on a similar subject to an article you’re looking for.

We then also have a second layer very similar to Amazon’s “people who bought this also bought…”, where we look over the history of users who used a specific resource to find common resources. These are then added to the mix and the rankings are tweaked accordingly, providing a human twist to the similar items by suppressing results which initially seem similar but which in actuality don’t have much in common at a content level, and pushing results which are related but which don’t have enough terms extracted for Jerome to infer this (for example books which only have a title and for which we can’t get a summary) up to where a user will find them easier.

Third of all in recommendation there’s the “people on your course also used” element, which is an attempt to make a third pass at fine-tuning the recommendation using data we have available on which course you’re studying or which department you’re in. This is very similar to the “used this also used” recommendation, but operating at a higher level. We analyse the borrowing patterns of an entire department or course to extract both titles and semantic terms which prove popular, and then boost these titles and terms in any recommendation results set. By only using this as a ‘booster’ in most cases it prevents recommendation sets from being populated with every book ever borrowed whilst at the same time providing a more relevant response.

So, that’s how we recommend items. APIs for this will abound, allowing external resource providers to register ‘uses’ of a resource with us for purposes of recommendation. We’re not done yet though, recommendation has another use!

As we have historical usage data for both individuals and courses, we can throw this into the mix for searching by using semantic terms to actively move results up or down (but never remove them) based on the tags which both the current user and similar users have actually found useful in the past. This means that (as an example) a computing student searching for the author name “J Bloggs” would have “Software Design by Joe Bloggs” boosted above “18th Century Needlework by Jessie Bloggs”, despite there being nothing else in the search term to make this distinction. As a final bit of epic coolness, Jerome will sport a “Recommended for You” section where we use all the recommendation systems at our disposal to find items which other similar users have found useful, as well as which share themes with items borrowed by the individual user.

Delicious tea…

Everybody loves cups of tea (or, if you prefer, coffee). Boiling the kettle to make those cups of caffeinated goodness takes energy, something which we’re constantly trying to use less of. Which is why when I discovered some of the University’s energy data I knew what had to be done.

You can now take a look at the electricity consumption of various University buildings expressed as how many cups of tea you could make with the same amount of energy. There’s even a pretty graph which shows how energy use fluctuates over the last 24 hours.

Open data is good – it lets us throw things like this together in a matter of minutes rather than hours.

Where are you?

We love open data. In fact we love open data so much that we’ve got a whole site dedicated to it, and we’re constantly looking at ways of making our data easier to work with for everybody involved. We’re also constantly looking at what new data we can glean, scrape and make do really cool things; which is why I’m really excited to tell you that we’ve got some delicious new things available over our Locations API.

First of all, I’d like to stress that this data is in no way complete. There’s a major project currently underway with our colleagues in Estates to replace their facilities management software, which when it’s completed will massively improve the quality of an awful lot of our data. In the meantime, here’s what’s new in our buildings data.

For all the social media junkies out there, some Foursquare data. We’re now including Foursquare venue IDs for buildings where we know them, and our Foursquare curation team are constantly adding to our brand page, making sure venues are correctly located, categorised and full of good tips. You can take a look at Foursquare’s own API if you’re interested in mashing things together – I’ve already got an RFID Check-In machine.

Next, outline data and latitude/longitude. You may remember that a feature of one of our old APIs was some lat/long data which could be used to work out where buildings were in a physical space. We’ve now started taking that a step further, not only providing just latitude and longitude but also providing an array of lat/long points which draw the outline of a building! Drop the points onto a map and draw a closed polygon between them and you’re done.

Thirdly, related to the second one, you can now request that any call to /locations2/buildings returns its output as a KML file, just by setting the ‘format’ parameter to ‘kml’. This means that you can grab KML for all the buildings on the Brayford campus (as an example) just by requesting /locations2/buildings?campus=brayford&format=kml. KML files are directly compatible with Google Earth, and can even work directly with Google Maps, overlaying a map of our buildings which is generated on-the-fly.

Up next in the glorious race for data is updated documentation, better and easier access controls (and the long awaited application registration), and working with some colleagues in our Social Computing Research Centre to crack open some of our energy usage data.

Is Blackboard down? Now you can find out.

Today I’ve been looking over some of our stats for service uptime, and realised it would be handy if we could let you (the staff and students using them) know when things were broken.

Now you can, as I’ve just added another three of our core services (Portal, Email and Library Catalogue) and one non-core but useful service (Blogs) to our Pingdom monitoring system. They now join Blackboard on a brand new public status page. Even better, because we like being open with things like this, you can see the history of our monitoring as far back as it goes. For the new ones this means you can look over history back to today, but for Blackboard this goes all the way back to February.

Pingdom’s monitoring is from a variety of locations around the world, meaning it reflects ‘real’ availability and not just what we can see from our own internal network.

See what’s up and what’s down, any time, at stats.lncn.eu.