These are unedited transcripts and may contain errors.
PLENARY SESSION: 4 P.M., MONDAY, 16 APRIL, 2012:
CHAIR: Good afternoon, people. We will shortly start the next session, so, I'll appreciate if you can take your seats. Okay. We really need to get going, if we want to be on time for the social event as well.
So, we are going to have ?? this session is diveded into three ?? by the way, I am Filiz Yilmaz, I am speaking on behalf of the programme committee and we will chair this session together with Andre. So, this session runs until 6 p.m., and it's divided into three sessions: We will have two talks and then followed by a panel. The first talk is from Gorazd Bozic, and he will be talking about Kafka, what Kafka write about Google and clouds.
GORAZD BOZIC: Thank you very much. I will mainly talk about Google, Google, not Kafka, I'll just briefly mention Kafka. Now, I believe that this is going to be a slightly out of the ordinary presentation for a RIPE Meeting in several ways. First, the title is not what you are probably used to. Second, I will talk about two incidents that we have handled in the Slovenian computer emergency response team recently that are also unusual because we usually are dealing with computer breakings, we deal with BotNets, infections, Trojan horses and so on. These two are a little different.
I know that there are several Google employees present here. I have had a chat with a couple of them, and I asked them to keep an open mind about what I'm going to say.
So when did you last Google yourself? The thing of search engines has definitely changed the way we live our lives, return to Google for advice and suggestions. And it has become part of our culture, because we don't search the net any more. We Google for stuff. We don't Yahoo for it and we don't binge around. Although today might do that. We Google for things, and every now and then, I am sure everyone of us has Googled himself or herself to see how we fit into society, how others perceive us and how we relate to our peers. Imagine now that if you Google yourself and your company, the first thing that comes up in the search results is a porn site. Imagine now that you are currently trying to win a tender, a very important tender for your company, and it is quite reasonable for you to think that the people deciding whether you will get this job or not are going to Google your company and they are going to Google you personally. Now, free pornia tied with your name in that context is not good. Okay, but mistakes happen and mistakes can be corrected. Nobody is perfect and mistakes happen everywhere.
And there are two ways that we can correct this problem in a network security speak, there are two attack investigators that we can use here to get rid of this problem. First is the Google's safe search mechanism. That is the algorithm that determines what content is appropriate, which content is inappropriate to show generally to people when they are asking for something.
So, we suggested to our client, to the lady that asked for our help, that she report this as a safe search failure and we did that also ourselves. Now, our request, as you can see in this picture, was denied. Why was it denied? Because Google wasn't able to find this web page or image in their index. They suggested that we mistyped the URL, that this is not what we see. We checked, we double?checked, we triple?checked, we used different browser, we used different computers, different operating systems, we tried with different ISPs, went through Tore anonymous proxies, every time free pornia when you mentioned first name, last name, company name for that person, free pornia always popped up. I did mention two approaches. The other is where you can report that your personal data is being misused by someone on the Internet, and of course Google, as a search engine, indexes that. So, you report to Google that there is this misuse of of your personal data, and interestingly enough, you get no response from them. You get no reply, whether they have received this request, whether they will process this request, or that they have looked into this request and think it's rubbish.
So what what is Google telling us here? That the reality is not what we see. They are telling us we see the free pornia amongst the results and they are telling us, no, no, no this is not amongst the search results. Now, this message that the reality is not what we see together with personal and alienated feeling when they were going through clicking through these online forms that Google provides. Reminded me of my secondary school days in Franz Kafka, Franz Kafka wrote in his novels about surreal things and the Government bureaucracy that produces surreal situations. This is not Government. This is a big company, but we were getting this strange feeling also. Okay, but this is not just Google's problem. There is content placed on servers that Google indexes and although the safe search mechanism is not working, there is the source where this free pornia is hosted. So, we looked at the ISPs where this is located and came to cloud flair, now, cloud flair is a young company that provides reverse proxies, it's a kind of a start?up company, and immediately after communicating with them, we could see that the abuse and support functions were not the priority when they were building this start up. They said that they were not legally ?? that they were not legally liable for this content because the content is hosted elsewhere. Now this is a discussion for another presentation, whether if you provide reverse proxies and you are alert that had there is illegal content or some strange content with your customer, with which you have a contract, is this legally binding or are you liable or not, but we went, we said thank you, and we went to the company that's basically the source that's hosting the original free pornia site. That's LeaseWeb. We had a series of communications with LeaseWeb, a series of e?mail exchanges. First they said, we are going to notify our customer, thank you. We had a slight feeling they were trying to avoid some responsibility here, so we said well, maybe you should act more quickly, can we see your terms of service so we can inspect whether this is violation of your terms of service and whether you should act more promptly? They said, well, we rechecked and this is not our customer. We said wait a minute but you said that it is and we have the ISPs that we got from cloud fair, they said oh yeah, yeah, this is our customer and we will remove the site today. Which they did not.
In the meantime, we were poking Google with our standard requests via e?mail in order to get this damaging package off their search engine results. We wrote to investigations at google.com and security at google.com and we received no reply from them either. The problem was corrected ten days after we first reported it to Google. We still don't know why the problem was corrected. This is also an important piece of information. When you are doing incident response it's very nice to get some feedback on why the problem has disappeared, so you know in the future on how you are going to react.
Now, if this was the summer story, there is also the winter story. This is a screen?shot of a recording of the closed session of the Slovenian Government. This was not intended to be placed on YouTube. I think this is a disgrace for our country that this was made public on YouTube, but I'm not here to talk about that. I am here to talk about our experiences and briefly show how we dealt with it and how our communication with YouTube and Google went in this case.
Google owns YouTube and Google is a member of First. First is the form of incident response and security teams which brings together all the incident handlers in the world. And every incident response team that wants to be a member of first has to register an e?mail address where they receive incident reports and that is security at google.com. We use that address although, as you could see in the first example, we saw that we are getting no replies from them. Abuse at YouTube said this is an automated response to let you know that your message has been caught by your spam filter and won't be read. Well, with a little help from the search community, we did get to human contact two e?mail addresses for two human contacts at Google. We sent them an e?mail and they were very helpful and they said that they are going to forward those e?mails to the appropriate Department's and that the problem is going to be dealt with.
The next day, usually it takes 24 hours because we are used to that time difference between Ljubljana and San Francisco. The next day the videos were still there. Their view counts grew and the local media had a field day here we, then said, maybe there is another angle where we can approach that. We contacted the US cert, which is the Government cert in United States located at the department of homeland security. We immediately got a reply back from them saying that they have e?mailed security contacts at both Google and YouTube and requested that they comply with our requests. That also didn't help. We did explain that this is a matter of national security. Those videos did contain some information that was sensitive and classified, but for the next couple of days, we were still amongeying with various online forms, trying to take down those videos and we were sending a mail a day, so to speak, poking also the human contacts at Google saying please, can you do something? Can you at least say that you will not do anything?
Because, that is good information also; that somebody says we will not do what you ask us to do because... and you at least get ran and see if you can go from there. And we were slightly frustrated, I must say, and some of us, when we were discussing this, mentioned that this would be dealt with immediately, if this was a case of an uploaded video of Britney Spears, naked, dancing. So we said, wait a minute, what if we change this had? What if we say that this is a copyright issue? These are Government tapes, they were made on Government equipment, they were stored on government premises. Somebody stole those tapes, so it's Government property. And it's a copyright issue. So, how do we go about that one? Because, the contents are located in the US, we, of course, Googled around, the procedures that have to be used with the digital Millennium copyright act. Discovered that every company has to register an agent, a lawyer, who you can contact when you have a problem. Here is the URL where you can find a list of companies and DMCA agents, and we Googled around for the DMCA take?down notice template, and then we compiled our own template, we added some information, why we, as the Slovenian computer emergency response team, are legally allowed to make this claim on behalf of the Slovenian Government. And what do you know? Success. The next day the videos are down. Not available any more.
So, Ms. Tucker informed the YouTube that this is a DMCA intellectual property violation and that they need to be removed.
Not that now everyone is Slovenian hasn't seen those movies, it was in all the newspapers, it was in all the media, it was on the web, copied many times. But from an incident response standpoint, you could say that we kind of brought our ships to port, so we concluded that incident. Of course you will not be surprised that videos reappeared the next day under a different account, so, but at least now we know how to deal with that problem.
Well, we did take a short?cut: We tried going directly to YouTube saying well remember yesterday, these are the same videos that now appear under a different account. Can you take down those too? And we were again pointed back to online forms for general complaints. So we sigh a little, compile another DMCA take down notice, contact Ms. Tucker and then she forwards that YouTube and the videos go down and surprisingly enough they don't reappear. We did expect a little more of whack mole in this case, but all was quiet after that so we kind of succeeded partly because everybody was now familiar with us, the general public, all the secret service that is might care about our country, the foreign secret services, and so on.
So why did I choose to present these two cases here? I presented these two to the community of certs on a closed meeting and I got two types of responses. One was the 'me too' response, yeah, we feel the same way. The other response was, don't do this. It's not fair and you are not going to solve anything. I beg to differ here. I think that it is not wrong to feel that the Googles become so big that it thinks it is self?sufficient and it doesn't need anyone else. I think it needs to treat other players in the network community differently and that it needs to change a little how it reacts to the professionals in the community. Now, I know ?? I mentioned at the start that there are a number of people that work for Google. This is not intended as a critique of your personal work, not at all. But it is maybe a reminder that some of things are not all that good. Now, I know that Mike Hearn from Google is going to have the next presentation. He is probably going to explain why it is he thinks things happened the way they did and how they are correcting this stuff. But I think I would also welcome some criticism from you or questions or comments that you have on the presentations that I had. If nothing else, I hope that the DMCA part was useful so when you are confronted with some YouTube videos that you need to take down, you at least know where to look for registered agents.
CHAIR: Thank you Gorazd. We have some time for questions. So, please, if there are any questions, there are microphones. Maybe people are holding their questions until they hear the presentation from Google, so they can form a 360 degrees opinion on that.
GORAZD BOZIC: I also want to mention that I am going to stay after this Plenary in the evening, so if anyone wants to come and talk to me I am going to be there.
CHAIR: Thank you
FILIZ YILMAZ: We have Mike Hearn from Google, and he will talk about abuse at Google.
MIKE HEARN: I am going to talk about two things. I am going to be primarily discussing ?? my name is Mike Hearn and I work on the g?mail abuse team. Not YouTube or web search, we have lots of different products and abuse teams. I am going to primarily be talking today ?? I am going to tell you some stories from abusehandling [at] google [dot] com and some general trends that we have seen specifically in g?mail and I am going to talk about the specific instance of abuse report handling and for what it's worth, I completely agree that that experience of Kafka?esque and does not reflect totally well on us. I am going to talk about abuse report handling in the g?mail context. You can draw your own conclusions about what that means for other products.
.and also float some ideas about how are we can do a better job. I want to talk about spam because this is an issue that affects more of you. Hands up if you have been spammed by g?mail? How about if you or your users have been spammed by g?mail in the past few months. That's more ?? I was expecting more people to have received spam from us.
So, I want to start with a few stories. Let's go back to the beginning of g?mail. This is April fool's joke 2004, we launched this webmail service. It's invite only, so it's difficult for spammers to sign up. We leave the open invite stage in 2006, so two years later. Primarily due to hardware constraints, by the way.
So g?mail is attracted to spammers for a bunch of reasons. One is that we don't provide the sender IP, so it's harder for you guys to classify mail we send. This is deliberate. It's a privacy feature that we felt strongly about. We include the IP if you send through SMTP. So, a few trends came together to make a really serious abuse problem for Google at this time. One is that open sign ups really, they took away the invitation tree, which is a key thing we were using until that point to solve spam. And at this time, CAPTCHA solving like literally teams of people in developing countries like Bangladesh, they started getting organised and competitive and they take the price of solving CAPTCHAs down to one dollar per thousand solutions, which is very cheap. If you work in these countries and it's actually a pretty good job, you live on a salary of a few dollars a day and it's very safe and nice easy job, you can work from home. All this comes together to result in a situation where within months, over half of the mail we were sending at times is spam and g?mail is already two years old at this point, we are sending a lot of mail so this is a significant issue. So I am sure, you know, you guys, you work for network that is send spam and receive it. So I want to talk a bit about how we turned this situation around because pretty much we did, as measured by us. If you disagree, come talk to me afterwards of course.
At the time we saw spam as one problem. We had one team and we split the team in two and recognised abuse as a different issue from spam. And within a few years, that was about six years ago, and we got ?? we brought g?mail to the point where the days when half of all of our mail was spam are now gone as measured by the mail we receive back from ourselves. But we have a lot of ways of measuring this ourselves. And that said, some of you might be thinking well I got a spam from g?mail and you probably did because we send a lot of mail. We send 5,000 illegitimate mails per second. So, we defined spam as something that you got that you didn't want, which is a very brought definition. So sometimes you'll get stuff that you didn't want from g?mail, and, you know, we think this is a problem but at the same time you can't make everyone happy all the time. So if it's not really volume, then it's not necessarily something we want to tackle strongly because then no one would have an e?mail account. Everyone is going to be upset by something you send some of the time.
So, some of this might be generic and you can use it in your own networks and some of it might not be. We look at hundreds of different features of mail that's sent. Every time you send a mail we do a fairly complex risk analysis. There is machine learning and other things you are familiar with. The thing that really allowed us to turn this around, because mail is very easy to random eyes was a placement for the capture, they are too cheap to stop spam. We would detect account as being spammy or probably spammy. We would make them solve capital tours and the spammers would buy their way through captures and nothing happened. Phone verification is where the account is locked and you are forced to give up a password ?? sorry not password, a phone number. It doesn't actually have to be yours, you can use a pay phone if you like, any phone number will work, and then we send a code to the phone, you type it back in and you prove ownership of the phone in that way. This changes a problem for spammers from being, I will solve lots of captures to being I will obtain and control large numbers of phone numbers, which is quite difficult and expensive.
And we did a series of tactical operations again account sellers. So the move to phone verification of accounts we suspected of spamming made it harder to create accounts and keep them alive and as a result the black market started to specialise into shops where people would create accounts and sell them and spammers would buy accounts in bulk, use them, shut down and buy more accounts and so on. So we were slowly able to squeeze, put the nail into these markets, and of course because there is no point in sort of disabling accounts if you can just create more, we also used phone verification on account sign ups, so we try and detect some block sign?ups or make them expensive. If you have an issue with automated sign?ups on your networks, this is an approach that I can't recommend enough. It did wonders for us.
This is a screen?shot of one particular account seller. So, they still exist. We haven't driven them out of business entirely. The prices are here are per thousand accounts. You can see that g?mail.com accounts is about 70 dollars per thousand. Competing networks are like 5 or 6 dollars per thousand. This is a significant price difference and that's how we were able to achieve T even so, this guy has got a good deal. He is significantly cheaper than his competitors, most of whom are in the 120 to 150 dollars. That's the price range. Lightly paying people in countries where Sim cards are cheap and swap them in and out and make g?mail accounts that way. At these prices, you don't see big outbound spam campaigns any more.
That said, like I said we still send spam. So, one recurring problem we have had is the built inassumption that many developers at Google is if you pay you are not a spammer. Anyone who has been dealing with spam for a long time will know that this is not true and they will a happily pay for the ability to spam. After a few months if they haven't within shut down that's a good sign that they are not spammers. This just forces people, it moves the spammers on to payment instruments where they claim they will pay and then they don't because the card is delinquent, it's stolen, and so on.
So, this is just a specific instance of a more general case where you have ?? Google is a very large company. We have got over 10,000 engineers and product managers. They are thinking of using interface design, security, and scaleability and race conditions, writing software is already hard and when you throw in the need to think adversarially, it gets even harder still. So, this is a huge issue because Google's mail stream is so much larger than most other networks so the spammers are motivated to insert their mail into our stream. It's been a big issue for spam filters for a long time. The guys who go up against our network are much more motivated because if they succeed they can make a lot more money. It's as simple as that.
Let me give you a couple of examples. Some years ago, we say students like g?mail so let's make it available to universities, students will use it and they will stick with it after graduation and universities get to outsource their mail, which is great, that's what they want to do. So we say, okay, if you are a university you get g?mail for your students and you have higher sending limits and so on because spammers don't like to may tuition fees normally and people who implemented this said rather than manually, CSO universities, let the DNS admin to do it for us. It turns out that in at least one country, .edu domain is open and anyone can sign up and create a university. So you have thousands and thousands of fake universities appearing overnight and started sending mail. This sort of thing, when it happens it happens extremely fast and at extremely high volumes because they find these exploits professionally and they know that within about 24 hours of them starting to abuse one of these bugs, we'll have it figured. So while that window of time is open, they use very large BotNets and the usual things and they try and cram spam through the system. So, here is another one: you put a CAPTCHA on a form and usually that means it won't get spammed ?? like if you have a form which results in an e?mail going to someone, if you can control where the e?mail gets sent, that's an open invitation to get spammed. If you have a CAPTCHA on, it it's not in theory. We had this feature on Google maps where you can send mail. It allows to you share your location and it included a message and could you say which e?mail address to say send to. Basically, it's an open mail really and it looked okay because it had a CAPTCHA. What we didn't realise is that you could resubmit the same CAPTCHA solution over and over again for two minutes because that's how long the catch lasted until it timed out. Someone realised this and after experimenting a little bit got a large BotNet with several hundred thousands machines and a large army of people who would be working together and each bot would get a capture solution from a human and spam for two minutes until it timed out and so on. Literally, there is an army of machines working with a small army of people and we actually had to do an emergency roll?out of all of Google maps to kill this feature dead. Because it would have killed our ability to send mail the attack was of such huge volumes. Tens of millions of mails left within the span of hours before we pulled the plug on this. And that's only two examples of the kind of things that happen when you have a big mail stream.
So, I wanted to talk a little bit about the trends we see in 2012 and the things that are currently on our mind when we are not thinking about abuse reporting. Which I'll get to in a few minutes. So what have we been seeing? Recent times?
So, in April 2010 our world changed overnight. The bulk sign?up era was over. There was still spammers signing up and sending small volumes of mail and there always will be, just because of how we define spam. But the era of when they create 10,000 accounts is largely gone because of the phone verification and other things I mentioned. So, spammers realised that, actually, it was cheaper to start hacking people's accounts than it was to create them themselves. You would expect that this is not as easy as signing up for a web form. But you'd be wrong. We began to see ?? we suddenly saw, literally overnight, one particular gang trying over 1 million sets of compromised credentials every single day. These credentials being stolen from websites that have been hacked and the pass words, if they were hashed, reversed using video cards. Now it turns how the the password reuse for Google users is actually pretty good. It's only around 10 to 15 percent. Most people are smart enough if they go to low?security porn sites they are inventing new pass?words for them. This means that these are still authenticating to over 100,000 accounts today. They send spam. Hands up if you received a spam from a friend in the past year or two. So some you have seen the brunt of this kind of thing head?on. You get a spam and it comes from your friend. It's correctly authenticated and it typically contains a link to a hacked website which is redirecting it to where the spammer is advertising. Our reaction to this can be summed up as essentially, gosh, that's a little bit unexpected. What do we do now? And this was a huge, huge issue for us, right, because what actually had happened is the age of the password had ended. And it was never going to come back because the fundamental reality behind this is the Internet is extremely insecure, lots of people are signing up for websites that are extremely insecure. So, our solution was to say, from now on a password is no longer enough. It's a really good signal. It's an important one but it's not enough. If you sign up from a place and we are not really sure if it's if you, even if the password is correct we will not allow you to log in until you answer more questions about the account. Some of you if you have logged in to g?mail from here with a computer that wasn't here you might have seen this identity verification. If you have logged into Facebook, you might have seen one. It involves identifying photos of friends and a few other things as well. So we built something very similar because pass?words security wasn't going to improve. We also have this thing for two?step verification. Some people use it, the security?conscious guys at the back. This is a high?security mode for your Google accounts and other companies of course do it too where we deliver a code to your phone and you type it in.
Most people don't use it. So, we had to build this. And this screen shot was taken in around January of this year, so not that long ago. The top yellow?ish line is number of identity verification challenges being served per second and the bottom blue line is the number being passed, so that's our baseline false positive rate. Then there is a bunch of false positives between threes lines that you can't see. Now, you can see the top line here, this is a three day view. The bottom line is smooth. The top line is heavily distorted by spikes. These one of these is an attack against our user base which has been detected and deflected. You can see at this time there are multiple attacks against our users which correctly authenticate with the valid pass?words, multiple times per hour, often at very high speeds. We have seen attacks that go as high to 25 to 30 accounts being hacked per second.
So, the solution became basically that everyone drops what they are doing and we all work on hijacking, literally nothing gets done except trying to solve this problem because it appeared out of nowhere and we needed to come up with a solution very fast. Facebook are good, they have released statistics and it was interesting for us to see that it's basically comparable. What we are seeing is we have been able to get this to a point where it doesn't bother the majority of users. We have a 0.1% false positive rate. That's how many people pass the quiz out of all web logins. And we have managed to largely wipe out this problem for certain types of hijacking. Not all of them yet but I think we'll have that done by the end of the year.
Let's go back to the previous discussion and the reason we talked a bit about g?mail and stuff first because I wasn't totally sure what the presentation was going to be about, but abuse of port handling was obviously a component of that. Why is it so hard to tell Google that we suck? What is up with that. In particular, why is it so hard to tell that you say our users suck? It's not because we don't know. We see it all the time. So abuse@g?mail.com. What happens if you e?mail it? Not much. The reason is that it receives around 40 abuse reports per second at peak. The vast majority of these, of course, are automated. These reports come from other networks and we group them into feeds. So we have a feed from hotmail and a feed from Yahoo and a feed from various other networks as well and we classify these reports according to where they come from. These reports, because it's partly because of the volume and partly because of other reasons, they are automatically reviewed in almost all cases because it's just not possible to have people review this kind of volume because you have half the company investigating abuse reports and because of volume and other reasons, abuse report processing is hard. Let's discuss a little why do we find this hard? Why can't we search the web but dealing with mails you send to abuse@g?mail.com or whatever, why is that hard?
So, one problem is you know finding trusted feeds is not as trivial as it looks. We need these because we are taking action based on these reports. It's just like some random person e?mails abuse@g?mail.com it could be a legitimate report of some spam, or it could just be someone is pissed off and everyone on the Internet if they use the Internet has annoyed someone on it at some point. So you don't really want to just sort of accept someone, you know, someone said this account is annoying me, please shut is down. So we try and aggregate them. So lots of people are reporting the same account is bad, then that has some value. But what we find is that every time we add a trusted partner, we take on risk. The reason is trusted partners who are sending us report that we act on, sometimes they are incentivised to become untrusted partners. What does that mean? This means that, for example, we added a trusted feed some time ago from a well known security company, for example, and it was a phishing feed. So reported this, and we did some manual reviews and it seemed good. We continued to spot check and someone said hey you shut down my website and it's legitimate actually, it's some consumer advocacy and I am changing the details here, but it was like a website protesting against some big companies involved with GM foods. You get the idea. And they are being shut down because it appeared in this feed. We went to them and said, what's up with that? That's not phishing, and they said one of our customers reporting that to us as brand abuse and we said brand abuse, that's very interesting, because I don't think any such thing existed. What does this mean? And what had happened is they told their paying customers we can get stuff pulled from Google and obviously they then started inserting things which were not actually phishing things into the phishing feed. Often this caused very high profile false positive. They are often valuable users who we want to keep, in fact.
Even worse, if you have an abuse reporting mechanism and you put too much faith in it, what you find is people who really want to get content pulled start gaming the system and they'll start submitting large number abuse reports which look legitimate but actually aren't. And finally if you can create a trustworthy feed, that's not enough. Actually, you have to tell us about things that we don't already know. If we get a lot of reports about the same accounts, and one additional report which says the same thing doesn't add a whole lot of value, so we look for coverage in place where is we don't currently have it. The brutal reality is if you have a very small network, that means doing the work to constantly monitor the flow of abuse reports from you is probably not worth it because we find about the abuse from other places.
It's quite hard to avoid those other places. The major networks are all there. What's more, g?mail came very late to the webmail game and, as a result, lots of people forward mail, for example, my personal account forwards mails to g?mail. And, you know, so that means if you spam me you can't tell from my server setup that it falls to g?mail behind the scenes. But it does. We find out about mail even if you deliberately trying to avoid spamming Google users which some spammers do.
What about sending abuse reports? Some of you guys have users on your networks that aren't behaving and you would like to know about them. We get this kind of request from time to time. So, obviously if someone presses report spam on mail, then they are actually reporting it, so the abuse reports that we send contain basically copies of those mails. Users have a totally reasonable and totally valid expectation that people are not reading their mail. Especially if those of you have relatives or friends that are who don't have a good understanding of what computers are capable of, it's a very, very common misconception that people are reading their mail all the time. How else can spam filtering work, this is something we come across. It's really important to us that we can say, no, your mail is not being read. Employees don't read your mail. Abuse reports trickify the situation. People do click report spam on mails which is not spam. If you take a single abuse report, people report spam because their uncle sends annoying jokes and won't stop and they don't want to talk about it with him so they report his mail as spam. They report spam on mails half?way through a conversation, they report spam that's just, they are too lazy to unsubscribe and so on. So ideally, if we send abuse reports, you should be processing them automatically as well and you should have some reasonable privacy controls there to ensure that you are not just like reading people's as mail. Specifically, doing sanity checks. We do that occasionally, we dip a test tube in and shake it about and see what's there. If you are reviewing abuse reports, you are going to see private conversations fairly frequently. That's not okay and we have to make sure you are going to respect our use ears privacy in this way.
In general, what we look for are feeds that aggregate large numbers of users, feeds that have active anti?abuse teams behind them. Standard formats like ARF is pretty normal. Feeds which are automated which are not just going through people, like, check out this awesome mail and send it around the office and stuff.
So that said, I have told you about this is so hard, I made a load of excuses, what you want know is how can we make things better? The specific incident that have raised in the last presentation, I dug into some of this, not the YouTube site, because I didn't know what video it was, but the web search take down and we can discuss this later, which is fine. But ideas for moving forward on, like, a slightly wider scale, so what we can do a bunch of things. There is some technical stuff. ARF has this kind of concept of what phishing is. Because the action you take is different. In today's world we also need ?? this is a spam but it's not actually a spamming account, it's a hacked friend. Hotmail started sending abuse reports like this to us, but it's not really standard, it's an ad hoc agreement. Content abuse, so I am sure some you who deal with spam have had the case where the text of the message is randomised and the link points to a very popular website. It's difficult to classify the stuff correctly and we are basically helping spammers get past the spam filters which is not good. We want to find out about that. Then, perhaps, once you know like I said, everyone has worked on hijacking nonstop for years now because it was a most important problem. Know we are getting this under scroll, handling of abuse feeds and reports float back to the top of our radar. Finally, if we are going through all this work to ensure that you are sending decent coverage and all this stuff, it would be a shame if it was just specific to Google. It would be nice to have an industry neutral forum where people could send their feeds and distribute them out.
That's the end of my presentation. So thank you very much for listening.
CHAIR: Thank you, Mike. Well, let me open the floor for questions again. Maybe with a different result?
MIKE HEARN: If no one has got questions, that's fine by me. Is there a question?
AUDIENCE SPEAKER: You are just ignoring me. First question is: That sender IP, is there any chance that you are going to change that policy any time in the future?
MIKE HEARN: It's something we discuss all the time. Bluntly, I don't think it's going to change because the team pretty frequently splits down the middle on the privacy versus spam filter. If half of our outbound mail was still spam, the argument for it would be much stronger. One thing we found is IP address is no longer a useful signal for us, period. The campaigns that they were seeing with the hijacked accounts and the guys who were creating accounts and then selling them, one account per IP is the standard. So sending IP is less of a useful signal than it would have been anyway, so probably not.
GORAZD BOZIC: I would just like to explain why I think that it's maybe not the best thing. Because usually, what we deal with is some kind of scam, fraud or criminal act, that the police is investigating, and we are helping them and we have all indicators, we basically know that this is a local thing, that both of these guys are from here, are from Slovenia, and that includes in that processing and that investigation, it suddenly includes you, and we have to communicate with the central in the United States and then the police has to require officially, file a request that takes from three months to ?? we have experienced ?? two years to get any information back. And from our point of view, which may be subjective, but from our point of view, this is unreasonable, so you are incerting yourself in the investigation while with hotmail, Yahoo and all the other, we can quickly say this is local ISP so you need a court order to get the information from that SI P.
MIKE HEARN: It's not a policy we will change. The fact that it's taking three months to get that information sounds Kafka?esque, so, I haven't heard this. I know that we have a policy where police come to us and we give them the login traces if they have the right paperwork. You can ask the relevant guys why it seems to take so long.
GORAZD BOZIC: I don't think it's your problem buff the whole Interpol things looks very nice in the movies. Outside the movies, it's very bureaucratic and it takes a very long time. Now, that ties a little to the trusted feeds that you mentioned, and trusted partners. This is one of my points in the presentation, that we think that certs could be your trusted partners. Why? Because we don't generate massive amounts of complaints. We are low volume, and we deal with, if I say so myself, with serious cases, whatever the definition of serious, which means that you will not get a lot of complaints or reports from us, so, we could be your trusted partner. I think that it is not wise that we are not at the moment. I think ??
MIKE HEARN: Some certs are and they do send us these reports. So I was talking primarily from the perspective of spam and bulk abuse. Now from the case of targeted investigation and like one account is you literally care about that specific account, that usually goes to actually a completely different department who work with law enforcement and their job is effectively you know dealing with crime, well spam is crime in some cup trees, but dealing with crime instead of spam. Again we should just talk and I'll see if we can get more certs hooked up to that process because there are certainly some already.
GORAZD BOZIC: Thanks.
AUDIENCE SPEAKER: Hi, Malcolm Hutty from LINX. First of all, thanks, a great presentation, really interesting. Very quick question about account unlocking via telephone number verification. What's your policy on retaining that information, the telephone number I mean?
MIKE HEARN: Well obviously for the system to work we have to record that it was used, so we keep that, and then because spammers do obtain large blocks of phone numbers, we keep that around for clustering, but we don't ?? it's not like put into a central database where anyone in the company can just spam you. There is internal ackles on the data and things like that.
AUDIENCE SPEAKER: But do you keep the telephone number.
MIKE HEARN: We have to, otherwise we wouldn't know it's being used repeatedly.
AUDIENCE SPEAKER: You could have a marker saying the account has been verified by telephone.
MIKE HEARN: The way the system works is, if you want to unlock six accounts or whatever, you unlock one, the second and on the third we say no, this phone number has been used too many times. We have to actually count how many times it's been used.
FERGAL CUNNINGHAM: I have a question on chat from Miela in Ljubljana. He said right now there is a lot of talk about ransom crypt trojan here in Slovenia. I notice it used a g?mail account. Is it clearly visible in Trojan's ransom demand file? What does the g?mail team do in this case? Will this g?mail account be disabled and is it already disabled?
MIKE HEARN: Without knowing the account, I can't check if it's disabled. Yes, we do see g?mail accounts being used as drop boxes ?? I think you said ransom ware? We do disable these accounts, yes. In the ransom ware case, it's a bit tricky because you know you want users to get their data back but we do usually tackle that with ?? disable the accounts and we'll send the relevant cases to law enforcement as well.
AUDIENCE SPEAKER: Todd Underwood, Google. The sender IP thing, I wanted to ask a specific question. I was approached at a previous RIPE by someone who was proxying requests on behalf of the Iranian police and secret service, asking if we could please release sender IPs so that they could do a better job ?? problematic mails from within Iran. And this is actually true. Like, this actually did happen to me. And I want to point that out because I think that there is a somewhat pervasive belief in people who don't like this feature that there is no reason that this just hides things from legitimate good law enforcement agencies who have, you know, every right to access the information, but, in fact, it is also true that electronic communications protected by this kind of privacy enable people to have more open societies, and I think this case of the secret service or police of a country whose record you may not like on human rights, which could include the United States, not having easy access to this information may be a social good. So that's my thought. You will get the access to this to the Iranian secret police, yes?
MIKE HEARN: That's a question for the lawyers. I would imagine no. But, more seriously, you know the primary reason for that is people don't examine if they send a mail to someone like an ex girlfriend or something, then the location is revealed. It's a pretty straightforward case of expectations there. Any more questions?
Thank you very much.
CHAIR: And before we proceed to the second part of the session, let me remind you that apart from the applause, you can also raid the talks and Serge showed you how to do this through the menu, through the rating system, so please do. And with this ?? we do this little entertainment for you like in TV shows.
CHAIR: This is the first time so there is this mic juggling around. We are working on it. The last part of the session is RIPE NCC's last 20 years within the RIPE community, and we will have Axel as the moderator and his three speakers: Daniel, Nigel and Geoff.
AXEL PAWLIK: Another bit of housekeeping. A short announcement: tomorrow at eight o'clock, we will have a short meet, the Board breakfast BoF session in the blue room, so you are all very welcome, especially our members who want to talk to the Board and discuss some things.
All right, as you probably notice we have 2012, and there is actually so far in the future that we are carrying super computers in our pockets and they have actually finally introduced a flying car a couple of weeks ago at some car show. Okay, that is a couple of years too late, twelve at least, I was expecting it for the year 2000, others were expecting it for the 1970s, however it's so far in the future that it's, as Filiz has said, it's sort of the 20th birthday year for the RIPE NCC and we shout being so far into the future we would like to take a look back at where we were before, how that felt and where we are today, and of course where we will be in 20 years time.
So, just a couple of pictures of some years past and you'll see there is this guy with a beard, it's like the tourist guy of a couple of years ago. And he is in all of those pictures. I wonder who that is? Daniel.
DANIEL KARRENBERG: Well, if the technical people can put up the other slide show with lots of pictures while I am speaking. I just submitted it five minutes ago, so that was a bit unfair.
Anyway, many of you know I have been part of the RIPE NCC team for the past 20 years and Axel just revealed that so all of you should know. And some of you may even know that I am the only one on this team that has stuck around this long. So when the organisers of this anniversary festivity asked me to say a few words, it was a little difficult to refuse.
And the traditional thing to do on these anniversary occasions is to thank everyone and their parents. And today, I'll be entirely traditional. I will thank everyone in turn. And don't worry, I'll do it in groups so that we can finish in time for drinks.
So when we think about the RIPE NCC, the first thing that comes to mind is IP addresses, yes, autonomous system numbers Internet number resources. The RIPE NCC distributes those and we keep the registry for who is using those for the Europe and surrounding areas. And we do this according to policies developed right here in this room by RIPE. This is called industry self?regulation, and we have been quite good at it. In the nineties, many have predicted that we would run out of IPv4 addresses by the turn of the century, and mind you the turn of the century was 12 years ago. Others have predicted that the distribution of the resources would be grossly unfair either on the local, regional or global scale. Again, others predicted that unresolvable conflicts would be inevitable.
Now we are here, a dozen years later than predicted, and we are going to deplete the regional pool of IPv4 addresses in the next few months. Actually, quite accurate predictions, Mr. Huston, and low and behold, there is no panic, no desperate rush for the bank and no widespread discontent with our self?regulation. I think this is a tremendous success. We can be particularly proud of the success because we invented the concept of the regional Internet registry right here. The RIPE NCC was the very first one of these organisations and other regions have copied that later on.
In the 1990s some, and some in this room actually, proposed to just sell or auction off IP addresses; that was quite fashionable at the time. Yet, this community, the RIPE community stub only decided to proceed with self?regulation, and we do have quite a fair number of policies to prove that. These policies have not just appeared out of thin air. They have been developed here by you, volunteers from industry, who have the welfare of the whole community at heart, and yes, I think that's really true, we have heated discussions but I don't think we have destructive ones. We have discussions by people who actually care about the outcome.
Today, the financial sector is a frightening example of what happens in an industry that blindly believes in market forces and does not agree, more respect, essential ground rules. We are different. We agree on policies and we respect them. I believe that we, or you, were right in what we did.
Our self?regulation has supported the tremendous growth of the Internet. Both here in the RIPE region and elsewhere in the world. Personally, I believe that without prudent distribution of resources based on self?regulation, this growth would not have been possible. I know that all of us here are pragmatists who work for the health and success of the Internet. It is against our culture to congratulate ourselves for our own achievements but what better excuse can there be for this 20th anniversary of the RIPE NCC and the fact that we have almost depleted the IPv4 address pool in an orderly way.
I think all I thank all of you who part pate in RIPE and RIPE policy development in particular, and please join me in a big applause for this, our achievement.
Of course, the RIPE NCC does more than distribute and register address space; after all, it's a coordination centre. So the RIPE NCC is your platform to organise any activity that is useful for the whole community and needs a neutral and impartial place and team to just do it.
There is no time here to mention all the RIPE NCC achievements, so let me just mention a less obvious one, the DNS area. You all know that there is a big domain name industry organised elsewhere and actually the entertainment value of the gTLD process is quite nice, but that's not the way we want to do business here, is it? Anyway, but we here do DNS because it's essential to a healthy Internet and to the success of the RIPE community. What do we do? We operate one of the DNS route name servers, K?root. We provide support for all country TLDs, ENUM, and the like. We have helped to ensure real DNS software diversity by instigating and kick?starting development of the NSD name server software. That was a long time ago. We have contributed significantly to the development and deployment of the DNSSEC and that's something that's often underestimated, actually, there was quite a lot of this community's resources that went into actually making the standards workable, getting implementations out there, and so on. Others have taken over this, but we have really had a role there. We have also served as an incubator for centre the organisation of the ccTLDs in our region.
Even in the small DNS area, there are a lot of things that are there besides just this address space registration stuff. Such efforts need funding, and our community has risen to the challenge always by providing the necessary resources itself without Government or other third party help. Of course, community funded activities need formal organisation and the associated governance. And we have been pragmatic and successful in doing both. At first, we found an organisational home for the start of the RIPE NCC; it was called RARE, the organisation of national research and education networks, now known under the name of TERENA. That way, we did not get tracked from the real work by creating a formal organisation and arguing about it. RARE played a crucial role in getting the RIPE NCC started and RARE did so at a time when the eventual smashing success of the Internet was not widely foreseen. Many RARE members, in fact, were pursuing a computing network architecture, the OS I model, with considerable support from national governments and the European Community. So, it was by far not a natural thing for RARE to become the home of the Internet coordination centre. But RARE did, and that was very important. Almost all the indication research networks, even found a way to contribute to the operating costs of the RIPE NCC in the initial phase, and most still do. And at the same time, also the first few, very few, initially just one, commercial ISPs contributed. For governance, at first we had the contributors committee. Who remembers that? Anybody remember the contributors committee? Rob, remembers it, okay. That's good.
The name alone spells it out: First contributory resources then govern how they are used. Being in Ljubljana, I'd like to recognise a Slovenian ?? it's not Jan Zorz; there are other Slovenians too. Outside Slovenia, that's not well known. But this Slovenian played an important role for the RIPE NCC at the time that RARE was still the home of it. I am talking about Tomaz Kalin; he was secretary general of RARE, and I think I was the NCC General Manager, so it was all generals, and Tomaz was my boss, which is not an enviable position at any time. It was especially difficult at that time, since there was always a little tension between RARE and the RIPE NCC and between RARE and the RIPE community because of this protocol war that was going on. So, why being part of RARE, the RIPE NCC was very concerned about its independence, especially when it came to managing the actual work. Also, the RIPE NCC was growing. It was growing much faster than anticipated because the Internet was growing much faster than anticipated, so, we were growing much faster than RARE was growing and RARE's rules and method of operation weren't quite suited to that kind of growth. So there was ample opportunity for friction, and I see Tomaz already agreeing with me.
And he and I disagreed from time to time. But eventually, we always made it work, and Tomaz taught me, among other things, that the word "bureaucrat" can be said without a derogatory connotation and he has also thought me to value a seasoned bureaucrat was always on my side.
Thank you, Tomaz.
Later the RIPE NCC grew more and needed a more suitable organisational structure. This community actually, at the Dublin RIPE Meeting, for those who remember, decided to incorporate it as an association. Associations are very democratic organisations where the membership makes the important decisions. Everyone contributing resources to the RIPE NCC is a member and gets a say in how these resources are used. This is how we are organised today.
Let me recognise one more person who considered very much to RIPE NCC governance: Rob Blokzijl. Of course, you know him as Chairman of RIPE, but many of you may not realise his continued contribution to overseeing the RIPE NCC as well. And that you do not realise that it's not really surprising because Rob has had none, hardly any I think ever, formal role in RIPE NCC governance, a really small one, he says. Yet, every time, each and every time a crucial decision was to be made by the contributors committee, the RARE Council of administration, the TERENA executive committee, or, nowadays, the RIPE NCC Board, Rob was there and provided invaluable advice and wisdom. And sometimes Rob even forgot to check whether he was invited at all to these meetings. Without Rob, the RIPE NCC would not be what it is today.
So, after mentioning these two people, please join me in another big applause for all those who have served in one capacity or another on the various governance and oversight bodies, they all have contributed to make the RIPE NCC a success.
Last, but not least, and you have seen some of them there, they are the people who do the work who make it all happen: the RIPE NCC staff. Personally, I am still around because doing the work and making it happen is extremely rewarding. However, maybe the most important reason to stay around for me are the people at the RIPE NCC. They come from all offer the world, 20?odd nationalities, they are dedicated and hard?working and working in this team is fun. We have just completed an employee satisfaction survey, because the Board wanted to make sure that we are all happy, and the first thing this survey professional who presented the result said, when he started his presentation was: I'd like to work here. And I agree with him.
On the other hand, I am also very proud of what has become of the people who left over the years. Many have stayed in the industry and some even have come back. Of the first four from 1992, and they are right here, three are still in this industry. Anne Lord is with the Internet Society, supporting ISOC sectors, especially especially in emerging countries. Tony Bates, on the right here, has been at MCI and Cisco and today he is the CEO of Skype. I am proud that quite a few successful people in our industry have contributed to the success of the RIPE NCC and maybe, being part of the RIPE NCC team has contributed to their success as well.
A few more that come to mind are those who went to ICANN; for instance, John Crain, Leo Vegoda, Dave Knight, and, very recently, Filiz Yilmaz. Olaf Kolkman has been a really respected Chair of the Internet Architecture Board. Too many excellent and creative people went to Google, unfortunately. I could go on. I am very proud that the RIPE NCC team has always been dedicated, professional and well respected. So, finally, please join me and join the present RIPE NCC staff in a standing ovation for all RIPE NCC, past and present. You made it happen.
AXEL PAWLIK: Thank you. Thank you, Daniel. I think the first RIPE Meeting I attended, I remember you were up on stage talking about how much fun it was to be working there. So, not much has changed, or has it, Nigel? You want to give us an update on how it feel to be around today?
NIGEL TITLEY: So, I am Nigel Titley, I am Chairman of the RIPE NCC Board. Bearing in mind the Kafka?esque theme of this session, I occasionally wake up in the morning and think I have been turned into a gigantic cockroach. However...
20 years, the RIPE NCC has been here for 20 years serving the membership, and, as Daniel has said we began operations in April 1992. I have been around for most of that time; in fact, I distinctly remember going to Daniel when he still lived in the TERENA offices and banging on his desk and saying, I sent in an IPv4 request ? of course I said IP in those days ? two months ago, and I still haven't had it. Where is it? It still took another two months to get it, but, never mind, things have improved an awful lot since those days. 100 members in 1994; over 8,000 today, which is a wonderful rate of growth. It was, as Daniel said, initially set up as the network coordination centre, it was the first regional Internet registry, often copied, never equalled, and it's still performing those functions today.
It's a strong registry; that's probably its main function. It allocates Internet number resources and it makes sure that those numbers are accurately registered. We, in theory at any rate, know who owns or uses any particular IP addresses, any AS number, and, in order to do that, the RIPE database is maintained and takes quite a lot of resource.
Those are probably the main functions but there are a lot more.
Daniel has already mentioned K?root. RIPE NCC has been running the K?root DNS servers for some time now and doing it with instances all over the world.
ENUM: Who knows what ENUM is? My goodness, you should be ashamed of yourselves. It's the system for mapping telephone numbers into IP addresses and the RIPE NCC performance the root function for that under contract.
We, of course, support the RIPE PDP, that's the policy development process. It needs an all of lot of Secretariat support. You guys do the work but somebody has to right everything down. That's done by the RIPE NCC.
Training courses: Tremendous amount of of training gets done.
And RIPE meetings. Even though we only have two a year, consider the three we used to have, it still takes a hell of a lot of organisation and I think they do it pretty well. Things mostly work.
Regional support: Regional support is something that the RIPE NCC is really only started doing in the last few years and it's aimed to take the RIPE experience to places that wouldn't normally get it. So, for example, we go to Russia, we go to the Middle East and we are starting on a south east Europe type regional meeting. And then finally and probably most tediously, we actually defend the bottom up self?regulation model with governments and regulators and law enforcement. Most of whom have no idea what bottom up is, and we explain it to them time and time and time again. And sometimes the idea gets through.
We represent the members' interests, again defending the bottom up process, through a number of fora: WSIS, IGF, OECD and WCIT. In particular, the IGF meetings take an enormous amount of effort and cause an enormous amount of brain ache, we also coordinate with our other industry bodies, as you can see bow there. The other RIRs, ICANN, IANA, the IETF, the IAB, ISOC, and there are more.
So, what are the important issues for today? Well, it's the main issues that we have always had: Strong registry, accurate data, legacy address space, is starting to be a problem. The old poisonous swamp that was left over from the pre?RIR days when you could get an IP address by sending off a self addressed envelope. Still trying to clear up that mess. And finally, of course, and probably most importantly, coordinating the technical community for the good of the Internet. Something that's not always apparent.
So, how are things changing? Well, we live in a changing world. You may have noticed that IPv4 is running out. IPv6 is being adopted with conspicuous success, I might say, in Slovenia, apparently. People, governments in particular, have noticed this Internet thing. I mean it's only about around for what, 35 years, 40 years now and they think it actually might work, it might be useful, good stuff, so they want to regulate, it as governments always do when they notice something. It's a changing world. We have to deal with it. Certification, not everybody out there shonest like they used to be. Certificates help us to do something about that.
And finally, it's very useful to know what the Internet is doing. And the RIPE NCC spends an awful lot of time and resource actually measuring things and presenting it openly to not only its members, but to the Internet at large.
So, what are we really doing? Well, in order to get the accuracy requirement, we do increased audits. You, as RIPE NCC members, as LIRs, you will probably have noticed that it's getting ?? you get asked more questions when you actually ask for IPv4 or IPv6 or autonomous system numbers. That's part of making sure that the accuracy of the registry is as high as we can possibly make it.
Coming out of the trustedness, out of the certification, comes the need to establish contractual relationships, because, in general, these days, if you don't have a contract with somebody, you don't really know who they are. Part of the process of forming a contract actually involves in working out who the person you are dealing with is. So, that is, unfortunately, something that's started to happen. We have already talked about increasing regional support and outreach.
So, what else is happening? Well, IPv4 is running out. As you may have noticed, as I said. So, one of the things we are doing is monitoring and reporting on the remaining pool. Not that there is much of it left. It looks like it going to run out sometime in August, Geoff will no doubt tell me ?? August the 12th ?? okay. I hope everyone has that in their diary because that's when IPv4 runs out in Europe. I hope you have all got IPv6 all up running and ready, either that or lots of big non?operational NAT boxes.
One of the other things we have specifically been asked to do over the last few months is make sure the activity plan and the budget are very closely aligned. The activity plan for those of you that don't know is a statement that the RIPE NCC puts out every year to the membership for discussion saying exactly what it will be doing the next year. And one of the defects in the activity plan in the past is that it hasn't actually had a price tag on each of those activities. That's coming in. This next activity plan will have a price tag for every activity so you'll know exactly how much of your money is getting spent on every activity.
And, of course, we have done IPv6 stuff ad nauseam. Websites, training courses, policies, the lot. If we could possibly have done it, we have done it. And still, IPv6 doesn't seem to be catching on, except in Slovenia, of course.
What else? Well, developing information services as RIPEstat, which let's you find out lots of stuff on specific Internet numbers. There is Atlas, which is Daniel's brainchild, which is collecting state of the Internet and measurements from across Europe and beyond. There is RIPE Labs which is a sort of sand pit that you can go play in and present ideas and where the RIPE NCC can present ideas without it actually becoming binding or contractual. And finally of course, there is the LIR portal that you all use in your day to day job to get addresses, to register those addresses and so forth. The day to day job of the RIPE NCC.
As Daniel said, the membership has made the organisation what it is today. And we do need to hear from you. We can't guess. I have got many qualities, but being a mind reader isn't one of of them and I really need to hear from you guys.
We do stakeholder surveys, we do general meetings, and obviously we hear from you guys at this sort of meeting as well.
What sort of things can you do? What sort of things can you twiddle to adjust the behaviour of the RIPE NCC? Well of course, the bluntest one is board elections. If somebody isn't performing, if a board member isn't doing what you think they should be doing, you are of course free to not elect them. That's simple. That happens every three years. It's a bit of a blunt instrument but you have probably the best one you have got.
The activity plan: Which is, as I said, presented to all the members every year well in advance. The budget: Which, again is presented to all the members every year well in advance. And which you vote to accept or not.
And finally the charging scheme. Which you also vote to accept or not. It's your organisation. Please avail yourself of the methods of driving it.
Now what does the RIPE NCC have to avoid? Because there are things that it should avoid. Activities that serve no purposes. Every organisation does things that it probably shouldn't do. Let us know what you think the RIPE NCC is doing that it shouldn't do. And that sort of forms into activities that the members don't want. Maybe there are activities that are useful, but if you don't want to pay for them, we shouldn't be doing them. You are paying for all this. And what we must guard against are creeping features. You know the thing, adding extra knobs, bells, whistles, to services that maybe don't need them. We have got to guard against self preservation, all organisations /TKUFR from this as they get bigger, their primary aim in life is not to do the function they were intended to do, but to preserve themselves. We have to guard against this. And in the same sort of vain is empire building. The RIPE NCC must not become an empire, it must remain a membership organisation, run by the members for the members.
Okay. A quick run through what the executive board is. We are elected by the members. We are neutral. We are impartial. And we do know the Internet. We all pretty well work for Internet companies. But we very carefully put aside that particular hat when we get into the board room.
So, we will do what the RIPE NCC needs to do and what its members need to do. We have a difficult balancing act. We have to make sure that the RIPE NCC stays financially viable, but we have to do what's good for the Internet, what's good for the members, and also what's good for the staff.
So, we need your input. We are all of us around this whole meeting. We all of us drink coffee. We all of us walk the corridors. We all of us eat lunch, and we all of us end up in the bar. Collar us there. Tell us what you think. And we listen. We listen to your members. We listen to the community and in particular we listen to the staff. The staff should remember that. If any staff member err wants to come direct to the Board with an issue, please do so.
Finally, there is an informal executive Board BoF tomorrow, come and join us for breakfast. It's at eight o'clock, which is not normally a time I am up and at breakfast, but then never mind. It's in the blue room, which is somewhere upstairs, come and join us. I am told there is proper tea.
And finally, congratulations to the RIPE NCC for 20 years.
AXEL PAWLIK: Thank you Nigel. What I take from that is that everything stays the same and everything changes too. Against the background of all those Internet drafts always up and to the right, the one thing I do remember about my job interview I think in summer '99 is that the next hot thing was this IPv6 thing that we need to get out of the door, because we were immediately running out of IPv4, and they are still there, that's great. I love it, some stability. So, Geoff, what are we going to be doing in 20 years time? You tell us.
GEOFF HUSTON: Thanks so much for allowing me to join you in this rather unique celebration of 20 years with the RIPE NCC. My task, my brief this afternoon was to kind of look forward and to sort of change gear a little bit an to offer you, I suppose, a very personal view into where I think, as a group of folk who live, work, play, breath, inhale and are the Internet, where we are going in the next couple of decades.
I'm older than I look. I am a young youthful lad, but I do remember 1992. Did anyone buy one of these bloody things? This is a VAX 9000, it's a wonderful machine. It's a ripper. It had its own false floor, because it wasn't allowed to go on mere mortal floors. It had its own special air, because it was intolerant of the air that mere humanoids breath and there was a seat for the operator. I never knew what operators did but they used to sit on that seat and say no all the time. This was computing for real people.
Remember that? Because, that was 20 years ago. And the funny thing was, for digital, this was a complete flop. Did anyone work on one of these? Right, none of you bought it. It was the last of the great ?? sorry, there was someone ?? my condolences ?? this was the last of the great main frames, a sterling achievement, you know, this had ECL components, this had ?? it was intended to be water?cooled, but, in the end, they just couldn't get enough water around the thing so it was just massive amounts of air. A complete flop because the best things we build are right at the end, aren't they? So 20 years ago, what we all did was cluster around and feed those things in share specially built rooms. Oh, how we have changed. How we have changed.
So, let's wind forward to ten years ago, and understand that aside from apple, this industry has no style. This is what all of you had on your desks, and, by God, they are ugly, aren't they? This is the best that Compaq, I think, did, or Dell or HP or one of them. But we all had one, because all of a sudden computing wasn't a special thing. The computer wasn't in its own special room. It wasn't something the university bought one of. All of us had one. All of a sudden, computing was a consumer device. You didn't buy it from the computing department. You went down to the shop and bought the damn thing and you saw one of these, complete with its massive display, this huge speakers on either side, and, of course, replete with three?and?a?half inch floppies, because it was very advanced at the time and took an age. But that was what was going on even ten years ago, and I see now even over on the desk, none of you have them any more. None of you. All of you have now got laptops. How yesterday. How completely yesterday. Because, as we know from Mr. Apple, the new iPad is all about style, convenience and being incidental. Computing isn't something you do any more. It's not something you learn any more. You don't need a degree to master this thing, as long as you have got a working finger. All of a sudden, computing is part of how we live. You don't go to a special room. You don't talk to the operator. You are on the train going to work in the morning, you just bring it out because you are bothered and you want to amuse yourself. Computing is now a completely different role in life, because, like it or not, even today, close your laptops down because you are dinosaurs. It's a post?PC world even today. Isn't it?
When we meet again in five years' time, none of those lids will be open as lids. Because they'll all be over. We are now doing something else. So, where do from here?
You don't have to bother coming to Australia any more, because when you go there and you kind of get bothered and you have nothing else to do you catch the train from Sydney to Perth, and for three days that is all you see. So now you have seen it, you don't need to come, because that's the entire country; it doesn't change for three days. Anyway, five years out there is this kind of marker out there, it will take you five years to get that far, in five years what will the world look like?
A lot of the world in five years is visible today and the numbers are frightening, because silicone is just processed sand. And if there is one thing our industry, our silicone industry understands, it's mind boggling volume. Each year we make more than 3 billion computer processors, 3 billion little chips that compute. Last year, we formulated those up into 270 million Androids and iPhones and any of those mobile units, and, because we made a quarter of a billion of them, they were so cheap. You might have paid Apple hundreds of euros for that iPad; it didn't cost them anywhere near to make them because they made so many, it's under 100 euros, it's probably under 50, but Apple won't tell me. They run such expensive software. Nonsense, Android is free. I need new content for these. No, you don't, it's just the web, it's just Java, it's things we all do. All of a sudden, computing is incidental. It's a commodity. It's cheap, it's plentiful and you have got this industry with a massive production capability. 270 million units shipped last year. So let's just look at the most valuable company in the world today. Six hundred billion dollars, I believe, is its current valuation. So back in quarter three of 2010, it shipped 8.4 million of threes things. One year later, 20 million. That's volume. 42 carriers in 15 countries in one quarter. Even iPads, you guys love them. It has style, it has class and you suck them up so much. Quarter three profit in 2011: 7 billion dollars. Even Instagram looks cheap. But that wasn't it. Quarter four last year, even bigger. Because in that quarter, they shipped 37 million of these iPhones, 15 million of the iPad 2s, you should have waited. The iPad 3 is even cuter. Their profit for that quarter: 13 billion dollars. This company is more valuable than Exon. This company is now more valuable than Microsoft was at the peak just before the 2000 crash. Capital is now dominating this industry because it's panned erring to what folk want and it's not the only one. Google and their Android, volume is everything.
So, we are in the address business. And every single one of these devices is going to need one. And we can pretty accurately predict, from looking at Apple's volumes and the other Androids and Samsung and all the rest of them, what demand is going to look like. That's the red curve ?? sorry, the yellow curve, the up?and?to?the?right one. It's that curve. But over here are the addresses, and, all of a sudden, we can actually understand what's going on. That's the supply curve. APNIC has already run out in the Asia Pacific. This year, RIPE will run out in, what? Four months, 16 weeks. You haven't got your application in for more addresses maybe you should consider it's really going to happen, because we haven't got enough to go around. The shortfall is going to get huge. By 2014, life is going to turn pretty weird. And quite frankly, these curves are showing it, we are facing an enormous issue.
Within the next five years, we have to resolve it. Because, in five years' time we won't be shipping 300 million new mobile devices a year. This is a commodity businessment. We'll be up into the billions, because everything, from light bulbs to water sprinklers, to anything else, will actually have a little bit of a chip there. Smart grids and meter reading is just now. Tomorrow is a different world again. But, no, we are professionals, we don't plan ahead; we react, we like to dance right on the edge of the cliff. Have we done v6? No. The amount of penetration in v6 in most countries it sitting around in fractions of percent. Somehow, this industry just doesn't get it. So, all of a sudden, RIPE is going to run out the same way last year APNIC run out, while at the same time you are pushing out more addresses than you ever have in the past. You are going to slam into the wall at full speed. And a market is inevitable as a result. Because a whole lot of of folk need addresses and there are some folk who perhaps will think, maybe I don't. So, a market is a certainty. An after market, a redistribution. But we have got a problem. If I pay 100 million dollars for a /8, I don't want to run v6 the day after I have just invested 100 million dollars in that horse and cart. I have no interest in v6. If the prices get too high, all of a sudden folk will be investing enormous amounts of money in yesterday's technology and they will be ill equipped and actually unwilling to move ahead with v6. If the price goes too high, we have really got a problem, because some folk won't want to move, and others who can't afford that price have no choice. The one thing about the 1980s that made networking hard is that we had so many of them and they wouldn't talk to each other. And folk like Daniel and many others around the world spent all of their time building gateways to try and get the e?mail from network A to network B. Guess what you are going to have to do if the price really rises? If he we start to get to 1,000 dollars in address, the Internet will be the victim. Because at that point the network will fragment. The folk who pay that price have no interest in 6. The folk who can't afford it really have a problem. And the solution is that we won't all be in the same boat at the same time. If the price is volatile in these markets, if we actually stuff it up, again you are going to deter people. The incumbents who have addresses will be fine. All the new folk won't. What happens to all those new devices, the things that are beyond the iPhone, the chips I want to embed in my skin that ring every time I get e?mail. Scratch that, that's a really bad idea. What happens to them? If the price goes volatile we have stuffed it again. And interestingly enough, if we ignore markets and there is a low price, oddly enough no one has got much of an intent I have to move away. Again, we just freeze. So in the next five years I actually think we have a choice, and it's pretty ugly on one side, because the only way through if you don't go to v6 is squashing the entire world as we know it into http, IPv4 and carrier grade Nats, and it's really up to us, as an industry, to figure out which way we are going.
The frightening part of this, the truly frightening part, is that none of us in this entire global industry, worth trillions of dollars, have a clue. We have no ideas in five years' time which of those parts we are going to take as an industry. None. Which I find really scary, I don't know about you.
And partly it is because the driving impetus is actually market forces. It's actually the economics of this business that's going to determine that.
So that's five years. Bit of a challenge. Minor challenge. But can we look further out than five? What's going to happen after that? What about if we extend and sort of go a little bit further across the desert, still the same picture, and look at ten years out. There we are in 2022. I am optimistic. The rest of you might not be. I am optimistic. I reckon we'll have done v6 by then. Twenty years ago, DECNET died, and it died within six months. If the Internet is all v6 by 2022, there is no after life for v4. Oh this v4 will last forever is about the same thing as saying X 25 will last forever, SN A will last forever. It won't. Because there is no after life for dead technology. If we go v6, it's v6. And we'll all be there, which I kind of find is actually really quite encouraging, so this market in v4 is a short?term market and we have got to get it right, but it's not a 25 year investment in a new way of distributing address. It's short, and hopefully recollect it's quick. And the whole thing is about CGNs, they really are an aberration; don't spend too much money on them, you won't have them for much longer. So, by 2022, I really hope we are beyond this and we are in a different world. We are in a world that actually is the Internet vision. But it's not the world you thought. Because radio spectrum, which if you thought was scarce and highly valuable today, you haven't seen anything. Because, the Internet will be incidental. It will be where you are. So, going to a point where there is a socket in the wall to go to the Internet would be so yesterday. It will be what your parents did, not what you're doing, because quite frankly the Internet will be where you are. However, and the only way we can do that is with radio spectrum. So, as far as I can see, the spectrum will become incredibly valuable and under incredible contention and the price per wavelength in its highly populated city will be incredibly expensive and we are going to find a different model of actually wiring cities. Hong Kong is a really good example, because, when they wire you up and give you your broadband modum in your unit, they clone a 3G femtocell on it and you become the local server for the phone company, for the 3G phones that are around. What they are doing is slicing and dicing the spectrum into finer and finer and smaller and smaller areas to make that critical resource last for longer. So, all of a sudden it's not IPv4 addresses that are really valuable and really, really under contention. In ten years' time it's spectrum that will be the absolute gold, the absolute critical resource for the Internet of that time.
And clouds and data centres, oh God, they are very fashionable today, aren't they and anything that's fashionable dies eventually. The idea that computing and storage is so scarce that I can only hold one copy of it will be, by then, so yesterday. The things that have really worked are things that are massively redundant and massively duplicated. BitTorrent is one of the most amazing systems on the planet. It's so good that half of you keep on trying to connect using broken protocols like Teredo and you don't even notice your computer is doing is because BitTorrent is so massively redundant it works in the face of appalling failure. This is tomorrow's kind of protocol. This is tomorrow's kind of data model. That if data is that abundant, I don't need to store one copy. I'll store 20,000 of them. All of a sudden the idea that I need this special temple of data, the data centre, I think will go away and the whole idea that even computing is so if he nominally valuable I need to go to some data centre. No. I think computing will become a commodity as well, so the whole cloud and data centre thing, I suspect it will have peaked by 2022, and, like main frames, that's the end of it.
So we are almost there at Perth. We are over there at the far end, the other side of the continent. Almost nothing of 1992 is with us today. Does anyone have a 20?year?old laptop that works? Right, neither to I. Nothing in today's world will be with you then. Nothing. Certainly not your Apple Macs that I see most of you with. Nothing will persist. Why? We are not special. Martin levy showed me this. It came up ?? there is an URL ?? it's about the spread of devices, household things in the US from, initially, the turn of the century, 1900, looking up to about 2005, and it charts telephones, refrigerators and stoves, clothes washers; we started selling them into households around 1915, but by the time everyone had a clothes washer in their house was not until 1990 in the US. 60 years, 70, 80. But look at the microwave: 20 years. Look at the Internet: five years. We are much faster at adopting generations of technology, that the life my parents led is not life I am leading. But I suspect my children will go through multiple generations in one lifetime, and that gets really weird. Because all of a sudden, even the iPad has a lifetime of what, five years, before something else comes along, because all of a sudden, everyone will have one and get fixated with the next thing and this generations are going to get faster and faster.
So what's shaping our technology then? I spent ten years working for the telephone company. I think at the time when it died, because the telephone company used to be amazing. It employed the most people in any country. Even a country like Australia, 20 million people, 100,000 worked for the telephone company. They didn't care about consumers. They dictated the service. They told you what you could do. There was no such thing as a consumer product. The telephone company decided on the technology, rolled is out and that was it. But now we are in a different world and we are in a world where you and I, as consumers, shape technology. And everyone is trying desperately hard to make us use them. The whole story of Instagram is all about that instant uptake where 20 million people signed up in a small number of weeks. In a post?PC world, computing is a commodity. It's abundant. Storage is a commodity; it's abundant. And even communications is abundant; it's a commodity. It's incidental. All of a sudden it's about consumer devices and services that are going to shape the Internet's future. And all of a sudden how you compete is all about how you innovate. It's all about constantly moving forward and changing the shape of the world. I would hope that we don't spend our next five years trying to perfect the year 2000. It's over. It was a fine year, forget it. I would hope that we figure out that we are really moving on and the pace and the speed of that will be very, very, very much different. I would hope that inside all of this we still understand that underpinning this is actually one genius, it's actually the genius of creativity and innovation, it's the one thing I think working in this sector keeps me constantly entertained, amused, engaged and having a really fun time. And I hope you are too.
And I'd like to leave you with one very, very fundamental thought: the tap. All of you have one. It's in every home. It's everywhere. Think of the centuries of technological innovation that got us there. The dams, the hydrology, the pipes, the metallurgy to make it all happen. Astonishing. We take it for granted. Sooner or later, and I think quite soon now, the computing is going to go the same way; it's going to simply disappear and become part of our life incidentally, like many other things are as well. And I think that's terriffic. I think that's success.
AXEL PAWLIK: Thank you, Geoff. You sound like quite the single  naratarian there. Very positive in outlook. And on that bombshell, any questions, comments? Okay. Lots of stuff to think about and discuss over the beer that we will have in abundance, and we haven't seen the future yet.
Thank you very much.
CHAIR: If I may say before the beer, there are two BoF sessions happening between now and then, so please come to one of them if you are interested. Thanks.