Posted on Leave a comment

IP Geolocation: The Good, The Bad, & The Frustrating

Here at ARIN, one of the most common topics we hear about from our customers is geolocation of IP addresses. Common concerns include:

  • Hey, I got a new block and I am based in Canada, but the search engine I use thinks I’m in the United States.
  • My video streaming service won’t let me log in because it thinks I’m outside my home country.
  • This online location finder says I’m in another state. How do I fix it?

We must then break the news to them that, unfortunately, there is no master IP geolocation database. IP geolocation is done either via third party sites that provide that data (some for free, some for a cost) or by proprietary systems developed by a content provider for its own use. All of these use ARIN’s Whois data to make their geolocation decisions, and sometimes getting the correct information in Whois will help. Unfortunately, there just isn’t any way to guarantee that will work. We don’t have any control over how third-party sites gather and use our data nor how they arrive at their geolocation decisions.

This begs the question: just how widespread is the problem of incorrect IP geolocation? An academic article[1] published in 2017 provides some useful insight into the scope of geolocation accuracy. The study looked specifically at geolocation for routers (not end users) across multiple free/paid IP geolocation services. The basic conclusion is that while country-level IP geolocation is fairly reliable for US-based routers (> 95% accuracy), it can be significantly less reliable for other countries. Moreover, city-level accuracy was found to be far less reliable, with ~75% accuracy being considered the upper bound (or in other words, at best 1 out of 4 are geolocated incorrectly at the city level). In fact, the study specifically “recommend[s] users not to trust city-level accuracy in ARIN regardless of the database used” (emphasis ours).

Which brings us to the takeaways:

  • The good: IP geolocation at the country level is fairly reliable within the US.
  • The bad: It’s less reliable outside the US and not at all reliable at the city level.
  • The frustrating: IP geolocation is inherently a guessing game. It’ll never be perfect; the only question is how often it’ll be wrong.

That being the case, we encourage network operators to work with one another to help solve geolocation problems. There isn’t one silver bullet that will always work, but by asking other network operators for help (for example, on the NANOG mailing list) you have the greatest chance to either reach someone else who’s gone through the same process (and thus may have wisdom to share) or reach the site directly and work with them to correct the geolocation problem.

[1] Manaf Gharaibeh, Anant Shah, Bradley Huffaker, Han Zhang, Roya Ensafi, and Christos Papadopoulos. 2017. A Look at Router Geolocation in Public and Commercial Databases. In Proceedings of IMC ’17, London, United Kingdom, November 1–3, 2017, 7 pages.

The post IP Geolocation: The Good, The Bad, & The Frustrating appeared first on Team ARIN.

Posted on Leave a comment

What drives IPv6 deployment?

IP Geolocation: The Good, The Bad, & The Frustrating 1

It’s been almost seven years since World IPv6 Launch day on 6 June 2011. In those seven years, we’ve managed to place ever-increasing pressure on the dwindling pools of available IPv4 addresses, but we have still been unable to complete the transition to an all-IPv6 Internet.

Nobody predicted this situation when we first thought about the consequences of running out of IPv4 addresses. We all thought that the depletion of IPv4 addresses would, in a continuously expanding Internet, provide sufficient rationale for IPv6.

We have been proved right that growth of the Internet has been inexorable. Personal computers have jumped from the desk to our pockets and then leapt across to a world of managed devices. A census of the Internet’s connected devices would readily number in the tens of billions of devices. If they all needed a globally unique permanent IP address, IPv6 would’ve been an imperative over a decade ago. But that simply has not happened.

Instead, we’ve managed to completely redesign the architecture of the Internet.

Some might suggest we have undertaken this effort simply to avoid the transition to IPv6. Perhaps that’s a bit of an extreme view but the picture of IPv6 today is certainly puzzling. What’s going on?

Data can help answer this question, and at APNIC, we’ve been performing a large-scale Internet-wide measurement of the level of adoption of IPv6 since late 2011.

I won’t repeat the details of the measurement system here, but you can watch a presentation I gave at APNIC 44 describing the way we undertake this measurement.

At APNIC, we operate the measurement system using a sample rate of between 6 to 10 million endpoints per day, drawn from across most of the deployed Internet. We are able to analyze this data to produce a time series of IPv6 adoption.

Figure 1 – User adoption of IPv6 since late 2011 (percentage of total user population).
Figure 1 – User adoption of IPv6 since late 2011 (percentage of total user population).

When these measurements commenced in late 2011 there was very little in the way of IPv6 deployment, and the total deployment level was measured at 0.3% of the Internet user base at the time (that is, an average of 1 in 300 users could successfully use IPv6 to access a web object).

World IPv6 Launch day on 6 June 2011 saw the rate of IPv6 adoption rise from 0.4% to 0.8% on the day, but the long-term IPv6 adoption rate was still very slow. It was not until mid-2013 that we saw the Internet-wide IPv6 adoption rise above 1%.

A major change in the IPv6 deployment picture occurred over the two-and-a-half-year period from the start of 2015 to mid-2017. Over that period the level of IPv6 adoption rose from 3% to 15%, and the majority of that rise occurred in the first half of 2017. By the end of 2017, the level of IPv6 deployment was measured at some 18% of the Internet, but there has been no significant further movement in that number across the first four months of 2018. Just four months is probably an insufficient period to justify an assertion that IPv6 deployment has stalled, but the hiatus in the growth of the use of IPv6 is certainly a source of some concern (Figure 2).

Figure 2 – User adoption of IPv6 (2017 to mid-2018).
Figure 2 – User adoption of IPv6 (2017 to mid-2018).

Can we determine what factors might lie behind this recent slowdown in the growth of IPv6? Perhaps some pointers to an answer to this question lie in an examination of the opposite question. What are the common factors that have been behind the deployment of IPv6 in networks so far?

In this article, I’d like to look at some potential answers to this question and see if the accumulated data can support any of these theories.

Who uses the IPv6 Internet?

To start with it might be instructive to look at the Internet itself; in particular look at the national populations of Internet users. Where are the Internet’s users?

One resource I’ve been using is Internet Live Stats, which provides an estimate of the total population of Internet users per economy.

Today the five largest national pools of Internet users are in China, India, the United States, Brazil and Japan. Together, these five economies account for slightly more than one half of the entire current estimate of some 3.4 billion Internet users.

In terms of IPv6 adoption, it follows that if all of these five economies undertook a close to ubiquitous deployment of IPv6 they would be able to swing the IPv6 adoption measurement point to over one-half of all users. Obviously, this is yet to happen and the actual picture of IPv6 deployment is somewhat different. The most overt missing element at this national level, in this set of the five largest national Internet user pools, is China, where visible IPv6 deployment is yet to lift above relatively small levels of adoption.

If these existing IPv6 users are not in China, then where are they?

Of the estimated 513 million IPv6 users, let’s look at the subset of economies where the national IPv6 user population is at least 3 million, or more than 1% of the total IPv6 user population (Table 1).

Country IPv6 Users % of Total World Count % of National Users
India 242M 44% 52%
USA 117M 21% 40%
Germany 30M 6% 42%
Brazil 30M 6% 21%
Japan 28M 5% 24%
UK 17M 3% 27%
France 12M 2% 21%
Canada 7M 1% 20%
Belgium 6M 1% 58%
Malaysia 5M 1% 22%
Rep. of Korea 5M 1% 11%
Vietnam 4M 1% 8%
Australia 3M 1% 16%

Table 1 – IPv6 users per economy.

The economy with the highest IPv6 adoption level is Belgium. Other countries which high IPv6 deployment levels do not have a large population of Internet users in absolute terms, so missing from Table 1 are Greece (37%), Switzerland (34%), Luxembourg (33%) Uruguay (32%) and Portugal (25%).

This table might also suggest that within each economy the distribution of IPv6 users is uniform across all national ISPs. This is definitely not the case, and to sharpen our focus in looking at where IPv6 is deployed, we need to look at each Internet Service Provider (ISP) — Table 2 shows the 20 ISPs with the largest estimated IPv6 user populations.

Rank AS AS Name CC Users (est) V6 Users (est) V6%
1 AS55836 Reliance Jio IN 257,116,163 236,669,761 92%
2 AS7922 Comcast Cable US 48,845,229 35,701,856 73%
3 AS7018 ATT Services US 27,190,530 21,621,772 80%
4 AS38266 Vodafone Essar Ltd. IN 42,711,918 17,068,044 40%
5 AS22394 Verizon Wireless US 18,499,350 15,788,328 85%
6 AS45271 Idea Cellular IN 38,817,216 15,470,268 40%
7 AS21928 T-Mobile USA US 15,402,964 14,513,144 94%
8 AS3320 Deutsche Telekom ISP DE 23,392,392 14,118,853 60%
9 AS5607 BSKYB Broadband GB 13,714,642 12,821,973 93%
10 AS2516 KDDI JP 22,146,612 11,725,651 53%
11 AS28573 CLARO BR 24,184,316 11,048,081 46%
12 AS17676 Softbank BB JP 25,292,851 8,259,099 33%
13 AS3215 Orange FR 16,990,645 7,976,416 47%
14 AS20057 ATT Mobility US 14,455,761 7,967,862 55%
15 AS22773 Cox Communications US 11,537,814 6,224,326 54%
16 AS18881 Telfonica Brasil BR 14,125,727 5,844,652 41%
17 AS10507 Sprint PCS US 7,548,658 5,797,873 77%
18 AS9644 SK Telecom KR 6,912,564 5,654,940 82%
19 AS4713 OCN NTT JP 19,995,028 5,586,165 28%
20 AS31334 KabelDeutschland DE 7,456,144 5,340,966 72%

Table 2 – IPv6 users per ISP.

Three surprising numbers in this table are those of 94% adoption in T-Mobile USA, 93% in BSkyB and 92% in Reliance Jio. They are surprising in that prior to these deployments we had thought that an ISP deployment of IPv6 was only a part of the story; it was up to the connected user network to also use IPv6-capable equipment within their edge network.

The Comcast deployment level of 73% reflects this, in that while the ISP network is fully deployed with IPv6 support, there is still IPv4-only customer equipment in use. Upgrading equipment in the home or office takes time, and the 73% adoption level reflects this.

So why can these three networks achieve significantly higher levels of IPv6 adoption? I suspect that this is a reflection of the difference between mobile ISPs and fixed infrastructure of mixed ISPs, and even the dual stack technology used by the mobile ISP. For example, T-Mobile USA uses 464XLAT, where the communications system is an IPv6-only network, and all attached devices use IPv6 to connect to the network. IPv4 is a tunnelled addition, where the IPv4 module is found in the user device. When given the choice these devices have a very strong affinity to use IPv6 over IPv4. It would not be surprising to learn that a similar approach was taken in Reliance Jio and BSkyB.

However, these three ISPs are somewhat anomalous within the larger picture of per-ISP deployment, and it is more common to observe that within an ISP, IPv6 is not ubiquitously used by customer equipment. Within these 20 ISPs with the largest numbers of IPv6 users, we see a variation of IPv6 penetration between a low of 28% deployment (NTT in Japan) to 94% (T-Mobile USA). It is likely that the lower numbers reflect an ISP that has integrated a number of different consumer products into a single AS, and only some of the products have integrated IPv6 support as yet, or the numbers reflect a combination of customer equipment capability overlaid on the ISP’s network IPv6 capabilities.

The basic observation is that IPv6 deployment is by no means uniform and deployment levels are variable within an individual economy and even within an individual ISP. The question now is: Are there common factors behind these IPv6 deployments?

Theories of IPv6 deployment

Let’s look at the available data about the Internet, its users, and ISPs to see if there is some visible correlation between IPv6 deployment in the ISP and some other factor that we can measure. What we are looking for here is some evidence that IPv6 deployment and some factors are closely related.

‘IPv6 is only for the rich’

It’s not strictly necessary for an ISP to deploy IPv6 in order to provide a consumer Internet access product. There are apparently no significant services that are accessible only using IPv6, and as long as an ISP has sufficient public IPv4 addresses to service the activities of its customer base, then it is not forced to deploy IPv6. From this admittedly very limited perspective, the case could be made that IPv6 is a ‘luxury’ activity, as distinct from an ‘essential’ activity.

But is this the case? Is IPv6 an activity that is undertaken only by those ISPs who can afford to undertake an activity that is not strictly speaking absolutely required? Or, to put it more bluntly, is IPv6 only for the rich ISPs?

One way to measure the ‘wealth’ of an ISP is to look at the aggregate net wealth of its customer base. Using the current Gross Domestic Product (GDP) of each national economy, and the current national population estimates we can derive a GDP per capita for each economy. We can then use the estimate of the customer size of each ISP to derive a notional ‘wealth’ value by multiplying this customer size by the GDP per capita. The 20 ‘richest’ ISPs using this method are shown in Table 3.

The IPv6 deployment figures from these 20 ISPs show a range of IPv6 uptake from 0% through to 94%. Some 13 of these 20 ISPs have an IPv6 deployment above 35%. While three of the low-level IPv6 use ISPs are Chinese, there are also Korean, US, and UK ISPs in a similar position. The suggestion that IPv6 is only adopted by ‘richer’ ISPs is looking like a somewhat tenuous proposition these days.

Rank AS AS Name CC Users (est) Value (est) IPv6 Users IPv6%
1 AS7922 Comcast Cable US 48,845,229 2,783,982,672,084 35,701,856 73%
2 AS4134 ChinaNet CN 310,853,595 2,460,095,350,830 244,673 0%
3 AS7018 ATT Services US 27,190,530 1,549,751,447,880 21,621,772 80%
4 AS4837 China Unicom CN 161,782,709 1,280,348,359,026 109,825 0%
5 AS22394 Verizon Wireless US 18,499,350 1,054,388,952,600 15,788,328 85%
6 AS3320 Deutsche Telekom DE 23,392,392 988,562,485,920 14,118,853 60%
7 AS17676 Softbank BB JP 25,292,851 984,220,710,963 8,259,099 33%
8 AS21928 T-Mobile USA US 15,402,964 877,907,336,144 14,513,144 94%
9 AS2516 KDDI JP 22,146,612 861,791,112,756 11,725,651 53%
10 AS20115 Charter Comms US 14,901,389 849,319,567,444 1,528,640 10%
11 AS20057 ATT Mobility US 14,455,761 823,920,553,956 7,967,862 55%
12 AS4713 OCN NTT JP 19,995,028 778,066,524,564 5,586,165 28%
13 AS22773 Cox Communications US 11,537,814 657,609,246,744 6,224,326 54%
14 AS3215 Orange FR 16,990,645 642,144,437,130 7,976,416 47%
15 AS701 Verizon Business US 11,056,076 630,152,107,696 62,472 1%
16 AS209 CenturyLink US 9,668,638 551,073,691,448 24,719 0%
17 AS5607 BSKYB GB 13,714,642 546,089,615,156 12,821,973 93%
18 AS5089 NTL GB 13,230,998 526,831,878,364 967 0%
19 AS9808 Guangdong Mobile CN 59,428,828 470,319,744,792 17,133 0%
20 AS4766 Korea Telecom KR 16,463,111 454,085,527,602 385,156 2%

Table 3 – Top 20 ISPs as valued by GDP per capita.

Another way to look at this data is by using a scatter plot, comparing the IPv6 deployment level on one axis and the aggregate value of the ISP’s customer base on the other. This comparison is shown for the same 20 ISPs in Figure 3.

Figure 3 – IPv6 deployment vs ISP customer value for 20 richest ISPs.
Figure 3 – IPv6 deployment vs ISP customer value for 20 richest ISPs.

If there were a clear relationship, then we should see a clustering of the data into some band where higher levels of notional value of the ISP would correlate with higher levels of IPv6 deployment. Clearly, no such clustering is evident in these numbers.

It is also the case that the 20 richest ISPs do not contain the majority of the IPv6 user population. Some of the larger IPv6 deployments, such as Reliance Jio in India and Claro in Brazil fall outside this top 20.

The top 20 richest ISPs only contain a little over one-quarter of the IPv6 user base, while the other three-quarters of the Internet’s IPv6 users are served from ISPs outside this same top 20 rich ISP list.

If we take the 400 highest value ISPs and perform the same comparison of value against IPv6 deployment, the correlation between the two metrics is shown in Figure 4.

Figure 4 – IPv6 deployment vs ISP customer value for 400 richest ISPs.
Figure 4 – IPv6 deployment vs ISP customer value for 400 richest ISPs.

Within this larger data set, there is still no clear correlation. And while having access to funding to deploy IPv6 is obviously an advantage, it is clear that ISPs that serve all profiles of user populations have decided to adopt IPv6.

It seems that the data is saying that you don’t necessarily need to be an ISP for a wealthy and large base of customers to afford to deploy IPv6. Being rich may well help to support the IPv6 business case, but it’s not a strict precondition here.

‘IPv6 is deployed when the ISP is growing rapidly’

Another potential motivation for IPv6 deployment is the scenario where the ISP is experiencing a rapid growth in customer numbers. In this scenario, it may be the case that the ISP has not made sufficient provision in its IPv4 holdings for this growth and needs to find a workable solution. Using the IPv4 address market to obtain additional IPv4 address is the obvious approach and many ISPs have used this market to obtain more IPv4 addresses, but the approach is not without its associated cost.

While IPv6 is not backwards compatible with IPv4 and is not directly substitutable for a lack of IPv4 addresses, IPv6 deployment can ease the pressure on an ISP’s IPv4 address pools. Dual-stack hosts on a dual-stack network typically use a ‘happy eyeballs’ approach, where an IPv6 connection is attempted first, and IPv4 is used only if the IPv6 connection does not complete in time. This means that a dual-stack deployment will push usage to IPv6 as long as the users access services from sites also using IPv6. A rapidly growing ISP may well see part of a response to such rapid growth in the deployment of IPv6. The Reliance Jio network appears to be a good example of this approach.

How general is this scenario? If we could identify those ISPs who are experiencing the largest growth levels in recent months would we see a strong correlation with that ISP’s adoption of IPv6? Table 4 shows the 20 large (more than 1 million customers) who have experienced the highest growth rates over the past 16 months.

Only seven of these rapidly expanding ISPs have an IPv6 deployment greater than 10%. Of these ISP who have deployed IPv6 to this level, all are expanding their IPv6 deployment at rates even greater than the underlying ISP growth rate. It appears to indicate that where an ISP has commenced an IPv6 deployment, growth in the underlying customer base is accompanied by an even faster growth in the IPv6 deployment levels. But at the same time where there is no initial IPv6 deployment, the rapid growth of an ISP’s customer base does not appear to act as an incentive for IPv6 deployment.

Rank AS AS Name CC Users (5/18) Users (1/17) Growth V6% V6 Growth
1 AS9644 SK Telecom KR 6,846,142 1,521,612 350% 81% 55%
2 AS131445 Advance Wireless TH 4,657,758 1,047,844 345% 42% 42%
3 AS37075 ZAIN UG UG 3,776,883 1,021,996 270% 0% 0%
4 AS45271 Idea Cellular IN 38,917,738 11,901,462 227% 39% 33%
5 AS9605 NTT DOCOMO JP 5,840,898 1,826,417 220% 8% 8%
6 AS55836 Reliance Jio IN 252,882,243 79,115,818 220% 92% 19%
7 AS56048 China Mobile CN 4,632,749 1,479,558 213% 0% 0%
8 AS132061 Realmove TH 3,033,271 1,035,635 193% 0% 0%
9 AS56041 China Mobile Zhejiang CN 21,567,119 7,379,958 192% 0% 0%
10 AS38266 Vodafone Essar IN 41,404,353 14,332,740 189% 40% 40%
11 AS21928 T-Mobile USA US 15,522,746 5,725,701 171% 94% 17%
12 AS4761 INDOSAT ID 7,750,603 2,889,955 168% 0% 0%
13 AS56047 China Mobile Hunan CN 6,839,156 2,579,417 165% 0% 0%
14 AS10507 Sprint PCS US 7,582,025 2,936,890 158% 77% 49%
15 AS24389 GrameenPhone BD 5,202,927 2,018,349 158% 0% 0%
16 AS197207 MCCI IR 7,451,173 2,897,240 157% 0% 0%
17 AS33771 SAFARICOM KE 10,556,969 4,199,062 151% 2% 2%
18 AS23693 Telekomunikasi Selular ID 15,027,678 6,043,987 149% 0% 0%
19 AS15897 VODAFONE TURKEY TR 4,079,685 1,653,584 147% 0% 0%
20 AS5410 BOUYGTEL-ISP FR 7,359,006 3,015,444 144% 2% 2%

Table 4 – Top 20 ISPs ranked by customer growth.

We can look at the correlation between the underlying growth in the customer base and the growth in IPv6 deployment for the 400 most rapidly growing ISPs (Figure 5).

Figure 5 – IPv6 deployment vs ISP growth value for 400 growing ISPs.
Figure 5 – IPv6 deployment vs ISP growth value for 400 growing ISPs.

Once more, there is no clear correlation going on here. One sub-group of ISPs with growth rates between 10% and 210% show IPv6 deployment levels that rise as the ISP growth rises (as shown by the blue group in Figure 5), but a second group shows a constant level of around 40% IPv6 deployment for all growth rates between 2% and 350% (orange group). The majority of the data points are in the band where IPv6 deployment is less than 20% and growth spans 2% through to 280% (green group).

Once more, we cannot substantiate this theory from the available data. A rapidly expanding ISPs does not necessarily deploy IPv6 as a response to this rapid growth. Some ISPs do, but many others do not.

‘IPv6 is deployed when an ISP runs out of IPv4 addresses’

The development of the IPv6 protocol was a response to the projection of IPv4 address exhaustion. The rationale was that an operator would be strongly motivated to deploy IPv6 in order to avoid many of the issues of attempting to operate an IP service with insufficient IP addresses.

If this is the case, then we should be able to look at each ISP and estimate their level of IPv4 address scarcity and observe that those ISPs with the greatest level of address scarcity pressure also show substantial deployment of IPv6.

Table 4 shows the Internet’s 20 largest ISPs as measured by the estimated size of their customer base. For each of these ISPs, we’ve compared the number of users to the total span of advertised IPv4 addresses that are announced by this ISP. This gives us a notional value of the number of users for each visible public IPv4 address. The largest values — indicating the most extreme levels of IPv4 public address scarcity — are seen in Nigeria, and the largest ISP in that country has the ratio of 402 customers per public IPv4 address. Three of the Indian ISPs have ratios of 66 or greater.

The question is whether the higher the ratio of customers for each IPv4 address relates to deployment of IPv6 in that ISP? There are 10 ISPs in this list with an IPv6 deployment level of above 10%, but the address use ratios of these 10 vary from 0.34 (or three addresses per customer) to 70 customers per IP address.

Rank AS AS Name CC Users (est) IPv4 Addrs IPv4 Ratio IPv6%
1 AS4134 ChinaNET CN 309,309,607 108,180,224 2.86 0%
2 AS55836 Reliance Jio IN 252,882,243 3,593,728 70.37 92%
3 AS4837 China Unicom CN 157,000,163 56,262,655 2.79 0%
4 AS45609 Bharti Airtel IN 70,522,541 3,968,000 17.77 4%
5 AS9808 Guangdong Mobile CN 61,231,552 18,678,528 3.28 0%
6 AS7922 Comcast US 48,758,165 71,069,952 0.69 73%
7 AS38266 Vodafone Essar IN 41,404,353 601,344 68.85 40%
8 AS45271 Idea Cellular IN 38,917,738 584,448 66.59 39%
9 AS4812 China Telecom CN 35,767,235 8,605,952 4.16 0%
10 AS29465 VCG-AS NG 32,523,737 80,896 402.04 0%
11 AS8151 Uninet S.A. de C.V. MX 29,853,671 13,798,112 2.16 11%
12 AS4808 China Unicom Beijing CN 28,340,473 6,860,287 4.13 0%
13 AS7018 ATT US 27,078,307 80,679,680 0.34 79%
14 AS17676 Softbank BB JP 25,227,427 44,918,528 0.56 33%
15 AS28573 CLARO S.A. BR 24,281,752 9,138,176 2.66 47%
16 AS3320 Deutsche Telekom DE 23,467,950 34,140,416 0.69 60%
17 AS8452 Telecom Egypt EG 23,396,481 7,063,040 3.31 0%
18 AS9121 Turk Telecom TR 22,621,162 6,905,600 3.28 0%
19 AS2516 KDDI JP 22,149,880 18,765,568 1.18 53%
20 AS56041 China Mobile Zhejiang CN 21,567,119 2,939,904 7.34 0%

Table 5 – Top 20 ISPs ranked by estimated customer size, and IPv4 use ratio.

The table does not show a clear correlation, but once more we can take the largest 400 ISPs and use a scatter plot to match their IPv4 ‘use ratio’ (the number of customers per advertised IPv4 address) to the IPv6 deployment level. This is shown in Figure 6.

Figure 6 – IPv6 deployment vs IPv4 address use.
Figure 6 – IPv6 deployment vs IPv4 address use.

Most of these ISPs have IPv4 address use ratios of between 0.5 (or, inversely, 2 addresses per estimated customer) and 10 (that is, 10 customers per advertised IP address). However, there is no visible relationship between this IPv4 address use ratio and IPv6 deployment. The highest customer to IPv4 address ratios are in excess of 800 — experienced by a number of ISPs in Nigeria — yet there is no visible level of IPv6 deployment in those ISPs as yet.

This is perhaps the most surprising result. The motivation behind the timing of the development of the IPv6 protocol was to have a mature and well-understood technology platform long before we ran down the levels of IPv4 to the extent that not only are the pools completely exhausted but instead of deploying IPv6, many ISPs are deploying address sharing technologies.

There are good reasons why many ISPs have taken this path as a short-term business response. For many ISPs an investment in IPv6 does not offer immediate benefits to the ISP customer base. It does not reduce the ISP’s cost base, and if development funds are finite it is not unsurprising that many ISPs have chosen to concentrate on activities that offer direct outcomes, such as 5G in the mobile sector, or securing content arrangements in the broadband service sector.

IPv6 is often seen as a risk mitigation measure, and while address sharing technologies are capable of deferring the problem, IPv6 is not screaming for an operator’s attention in many cases. However, perceptions of risk can be very subjective, and while some ISPs are comfortable with deferring IPv6 deployment, others have already taken steps.

The salient observation here is that the level of address shortage being experienced by an ISP does not appear to have a strong influence on the perception of risk of a collapse of the IPv4 Internet.

‘IPv6 is deployed when an ISP’s competitors deploy IPv6’

The early days of the outreach of the Internet from the academic and research sector were seen as wild days! The industry was driven by entrepreneurs who saw a unique opportunity in challenging the entrenched monopoly of the telephone companies. The catch-cry of the time was innovation and a deliberate effort to distinguish these new entrants as being completely different from the ponderous conservatism of the telephone industry.

Inevitably, size creates inertia, and as the customer base of these service providers expanded their ability to move quickly was significantly reduced. The Internet business sector now shows behaviours that have much in common with the old telephone sector. One of the common characteristics in this form of environment is that the dominant operators in a market tend to act in similar ways, offering similar services and at similar prices to their customers.

What happens when one of the larger service providers in a market offers a new service? It is often the case that this provides sufficient impetus for other providers in the same market to offer the same service.

Does this same behaviour apply to IPv6? How many national markets exist where three or more of the six largest ISPs have an IPv6 deployment of 20% or more? The list is surprisingly small, as shown in Table 6.

Economy IPv6
India 58%
Belgium 58%
USA 44%
Germany 39%
Greece 38%
Japan 27%
Brazil 26%

Table 6 – Countries with combined IPv6 deployment.

The other more populous economies with a significant level of IPv6 deployment only have two or one of the largest six ISPs providing IPv6 services to their customers.

Does this small list of economies in Table 6 prove or disprove the theory that competitive pressures tend to push other ISPs to deploy IPv6 once IPv6 deployment has commenced in an economy? It’s hard to say.

What drives IPv6 deployment?

What can we observe about the drivers for deployment of IPv6 in 2018?

  • You don’t have to be rich, but it helps.
  • You don’t have to be growing rapidly, but at times rapid growth provides sufficient grounds to make the move.
  • You don’t have to be desperately short of IPv4 addresses, but again it may help.
  • You can do it by yourself, but it can help when your competitors do it as well.

When the IPv6 protocol was first designed, we thought of the Internet in the same terms as the telephone network — as a peer network. Every connected device was meant to be able to both initiate transactions and respond to transaction requests. Addresses were both a network locator and a persistent endpoint identifier. When we were projected to run out of IPv4 addresses the consequence was that the network could no longer admit more endpoints.

That was then. Today is very different. These days, it’s a client-server network. Clients do not need a persistent network-wide identity and only need addresses as and when they communicate with servers. Servers do not need persistent identity either these days, as the identity of a server is a name-based distinguisher rather than an address-based identifier.

We have positioned IP addresses in a different role, and no longer need to associate a unique public IP address with every connected endpoint. Address sharing technologies have allowed us to grow the pool of connected devices far beyond the number of unique addresses in the protocol.

How far can this client/server model grow while relying on IPv4?

How many devices can we cram into the IPv4 Internet before we break it?

Perhaps that’s a question we should not want to answer from experience.

This article was originally published on the APNIC blog.

The post What drives IPv6 deployment? appeared first on Team ARIN.

Posted on Leave a comment

Changes to Out-of-Region ROUTE, ROUTE6, and AUT-NUM Objects in the RIPE Database

Beginning in September 2018, RIPE is implementing changes to out-of-region route, route6, and aut-num objects in its database. First, users will not be allowed to create new route, route6, and aut-num objects for out-of-region resources in the RIPE routing registry. Second, existing route, route6, and aut-num objects for out-of-region resources in the RIPE database will have their “source:” attribute changed from “RIPE” to “RIPE-NONAUTH.” If you have routing objects in the RIPE database that reflect resources registered under ARIN, for example, those objects would be changed. The change to existing routing objects could impact some routing processes that use the “source:” attribute. RIPE has provided information on this change.

Does this affect me?

If you have created route, route6, or aut-num objects in the RIPE database for resources (IP addresses and ASNs), and those objects are for resources registered with ARIN, they will be affected.

What will happen to my objects?

RIPE will change the existing value in the “source:” attribute of those objects to RIPE-NONAUTH. These items will continue to exist in the RIPE database with this changed attribute.

How will this impact my routing?

Due to the complex nature of Internet routing, we cannot predict how the change to these objects may affect routing. It could affect routing queries if they are configured to use the “source:” attribute. However, routing queries that do not specify a “source:” attribute will receive both “source: RIPE” and “source: RIPE-NONAUTH” objects in their results, so that the impact is minimized.

What should I do if I am affected?

ARIN recommends that people with affected objects at RIPE consider creating routing objects for their resources in ARIN’s Internet Routing Registry (IRR). However, note that in the future, ARIN may implement changes impacting out-of-region objects in its own database.

How do I create routing objects in ARIN’s IRR?

Follow these steps:

1. Create a mntner object and set up authentication.
2. Create route, route6, or aut-num objects for your resources.

ARIN provides the following information to help you configure your objects in the IRR:

• Quick Start Guide to the ARIN Internet Routing Registry: Describes the first steps you need to take to configure IRR objects.
• Using the IRR: Describes how to create, edit, and delete IRR objects.
• IRR Templates: Provides descriptions of object fields in the IRR templates.

Where Can I Get Additional Help?

• For help with ARIN’s IRR, you can contact our Registration Services Help Desk.
• For help with RIPE’s registration and routing database, visit their Contact page.

The post Changes to Out-of-Region ROUTE, ROUTE6, and AUT-NUM Objects in the RIPE Database appeared first on Team ARIN.

Posted on Leave a comment

IPv6 Addressing Plan Design for Service Providers

IP Geolocation: The Good, The Bad, & The Frustrating 2

Clearcable IPv6 Case Study

Clearcable has been involved with IPv6 since the early 2000s when commercial products began to emerge. In the early days, it was nothing more than testing, experimentation and lab use; however, by 2008, we had set out on a campaign to start educating our service provider clients on IPv6 and need to start implementing IPv6.service providers take ipv6 seriouslyConvincing them was a difficult task, and we did not make much headway; however, there were a handful of service providers where we had the necessary approvals to proceed. We began by having each client ISP register their own /32 direct allocation from ARIN (the standard ISP allocation), and then we configured IPv6 for each client ISP in their core networks. This often meant firmware upgrades, and back then if we were able to get a BGP transit peer we did, or via IXP members that also had IPv6 and/or tunnels with Hurricane Electric.

In 2010, we launched our SOE4 private cloud platform and it was designed from day one to support IPv6. When possible, as those platforms were rolled out, we ran HE tunnels for IPv6 access and as more and more service providers obtained native IPv6, we migrated the server platforms over to native IPv6.

It was also in 2010 that I presented at our annual Clearcable Technology Summit about IPv6, covering the background, driving forces, and how to. This spurred on some additional interest, but it wasn’t nearly enough. We continued promoting IPv6 in collaboration with industry organizations and vendors since then, and we also developed implementation strategies, use-cases, and supporting infrastructure such as our IPv6 transitional appliance for DNS64 and NAT64 services.

We also knew that we had to design a standardized and sensible IPv6 addressing plan to help aid service providers in the deployment of IPv6. We built two tracks, one for service providers and another for content providers/enterprises.  Our main goal with respect to addressing was to provide a framework that helped automate address administration and minimize future renumbering.

main goal ipv6 addressing

The key concepts;

  • Simple and Sensible
  • Maintain nibble boundaries whenever possible
  • Maintain largest possible aggregates for routing specs and security ACLs
  • Provisioning of new addresses derives from existing system properties
  • Minimize future renumbering

As part of our IPv6 address plan design we developed a standard access technology map for our small to mid-tier service providers, outlined below:Clearcable IPv6 Address Plan Standard Access Technology MapLooking specifically at DOCSIS as an example, we have a /34 reserved. From this /34 we allocate a /36 for a region or PoP site and then assign individual /40’s to each DOCSIS CMTS (16 total for a /36). If we need more access technology selectors we increase the number of significant bits in front. If we need more devices within a single access technology, we extend the netmask to combinations such as “A (1010)” in DOCSIS. So now both “8” and “A” mean a DOCSIS CMTS, but they are still able to aggregate under a /34 (i.e. we do not have to enumerate two separate /36 blocks).

We also developed standardized end user assignments;

  • /56 = Residential/SMB
  • /48 = Content Provider/Enterprise
  • /64 = Service Provider infrastructure
  • /64 = Technology independent
  • /128 = Loopback

From a residential user perspective, a /56 block is roughly equivalent to an IPv4 /20, with 8-bits of usable addressing space which equals 256 LANs. Typically only network 00 will be used, for example 2001:db8:1234:5600:020c:29ff:fe0c:47d5/64 however more advanced setups will use 01, 02, etc. with one /64 per interface such as, 1000BASE-T, WiFi, security, IPTV, etc.

Sample delegation schemes are below;


  • 0x8 = DOCSIS
  • 0x0=CMTS#1
  • Residential Subscriber ID=0x3456


  • 0x4 = GPON
  • 0x03 = OLT#4
  • Residential Subscriber ID=0x456

In the case of a VLAN based deployment, the VLAN Identifier (VID) is 12-bits long and we map the VID (in hexadecimal) to the low 3 nibbles reserving 0 in the high nibble to designate VLAN mapping is being used.  For example :0678: where 0x0 indicates VLAN style mapping and 0x678 represents VLAN 1656 (decimal).  An example VLAN based deployment for service provider infrastructure would be 2001:0db8:0000:0678::/64 where “0000” represents internal infrastructure and the VLAN ID is 0x678 = 1656.

Beyond the standardized technology map and end user assignments, two other important design decisions in our address planning were to use sparse based allocations and to follow RFC 3531 for Flexible Bit Assignment.  This allows us to defer fixing the position of network boundaries as long as possible to keep aggregates large. Bits are allocated from the left in the left field when making sequential assignments, for example, 0b0000, 0b1000, 0b0100, 0b1100. This allows us to grow either the number of routers or the number of networks when required. R00N: can become :RRRN: or :RRNN: or :RNNN:, as an example.

We have also built into our IPv6 addressing plan provisions for link-local, multicast, NAT64, loopback, point-point and other special addresses.

Additional best practice considerations are below,

  • Follow RFC 5952 for address representation
  • Prefer SLAAC for address synthesis
  • Sparse allocation accommodates growth
  • Use a consistent generic delegation schemas
  • Use RFC 3531 to delay subnetting decisions
  • Recognize the purpose of special addresses

IP Geolocation: The Good, The Bad, & The Frustrating 3

In summary, no matter what IPv6 addressing plan is decided upon, the key is to have one and that it be well defined in advance, shared with all key stakeholders (including training), and then followed.

Industry organizations like ARIN, the Internet Society, and others have been instrumental along with many content providers such as Google and Facebook in advancing IPv6. Over the past few years we have finally seen service providers take IPv6 seriously, and now 10 years after we began our push for IPv6 adoption, nearly all our service provider clients have IPv6 deployed to some extent with active plans for continued advancement of their IPv6 implementations.

The post IPv6 Addressing Plan Design for Service Providers appeared first on Team ARIN.

Posted on Leave a comment

Have You Heard About NRPM 4.10?

You may have heard that ARIN’s IPv4 free pool ran out over two years ago. Since then, many organizations have decided to #Get6 and transition to IPv6. However, some of these IPv6 networks find they still need a small block of IPv4 addresses to help get their IPv4 customers and service to/from IPv6-only networks. Because the IPv4 free pool depleted, it’s not as easy to receive IPv4 addresses from the free pool these days. In fact, many organizations wait for months on the IPv4 Waiting List or they receive IPv4 addresses via Specified Recipient Transfers.

Did You Know?

The ARIN community recognized early on that IPv4 would deplete and that network operators would need to adopt IPv6. During this early period, the community enacted a new policy that would place a /10 of IPv4 addresses into a reserved pool specifically to be issued to organizations that adopted IPv6 but still needed to transition their customers and services between IPv4 and IPv6. This policy is outlined in the “Number Resource Policy Manual” (NRPM) section “4.10 Dedicated IPv4 block to facilitate IPv6 Deployment” (NRPM 4.10). This policy, reads in part:

“Allocations and assignments from this block must be justified by immediate IPv6 deployment requirements. Examples of such needs include: IPv4 addresses for key dual stack DNS servers, and NAT-PT or NAT464 translators.”

You can qualify for IPv4 addresses from this reserved pool if your organization has IPv6 space already registered, whether directly from ARIN or as a reassignment from your Internet Service Provider (ISP), and needs a small block of IPv4 addresses to deploy your IPv6 network.

So now you want to submit a request, but aren’t sure how?

If you’re new to ARIN, follow our Quick Guide to get set up in our system with your User Account, Point of Contact Record (POC), and Organization Identifier (Org ID). Once set up, submit an IPv4 request and ensure you select the proper form (ISP IPv4 or End-user IPv4). You will need to click the radio button to indicate you are requesting under NRPM 4.10. In the justification section, tell us how you will be using the requested IP address block to transition to IPv6 (e.g. dual-stack, NAT464, etc.). We’ll review your request and let you know if we need any additional information. Once we’ve verified your organization meets the qualification requirements, we can approve your request.

Other things to note

  • If you already have IPv4 addresses directly registered to your organization, policy requires that you demonstrate that no other assignment/allocation is available that you could use to facilitate IPv6 deployment.
  • What if you have already been assigned/allocated IPv4 addresses under NRPM 4.10, but you find you need more? You can simply submit a new request if your previous assignment(s)/allocation(s) continue to meet the justification requirements of NRPM 4.10:
    • your previous assignment(s)/allocation(s) were issued to your organization at least six months prior,
    • the previous NRPM 4.10 block(s) are being efficiently used, and
    • continues to be used for IPv6 transition purposes.

Have any questions? Give our Registration Services team a call at 703.227.0660 or submit an Ask ARIN ticket from within your ARIN Online account. We’ll do our best to help you and your customers connect to the whole Internet.

The post Have You Heard About NRPM 4.10? appeared first on Team ARIN.

Posted on Leave a comment

Make IPv6 Work For You, Not Against You

IP Geolocation: The Good, The Bad, & The Frustrating 4

Watch Communications IPv6 Case Study

Watch Communications started as a telephone company back in 1902; I worked at Omnicity, where we had built a small IPv6 network prior to being acquired by Watch Communications in 2010. When we came on board, Watch Communications said they wanted to look into doing IPv6 too. Today we have every office IPv6-enabled and push IPv6 to our hosting customers. On the Internet service side, our biggest hurdle has been trying to justify the costs in upgrades for equipment and investment in time to train networking staff and support staff on IPv6 and how to troubleshoot it with downstream customers.IP Geolocation: The Good, The Bad, & The Frustrating 5

Every work station can get to the Internet over IPv6

The way we started our initial IPv6 project was to make every office fully IPv6-enabled. Some offices have native connectivity to an IPv6 leg of our network, and others we provide IPv6 through a VPN tunnel back to our corporate headquarters. One hundred percent of our data infrastructure is IPv6-enabled. Every server has at least one IPv6 address as well. For office locations that don’t have native IPv6 connectivity out to the Internet, we’re pushing the IPv6 default route as well.

Taking IPv6 to hosting customers

In 2013, we started pushing our hosting customers to have IPv6 addresses for all of their websites and email hosting. Today 90% of all our hosting customers are actively using IPv6 in their communications, whether they realize it or not. We do full IPv6 connectivity outbound through email.  As soon as customers sign up for any kind of website hosting, even if they’re on a shared IP with the hosting solution, they get IPv6 as part of that automatically.

The first thing we needed to do was get our server infrastructure connected to IPv6 so we could log in and manage the servers. Once the servers could get out to the Internet on IPv6, we made sure our DNS servers were accessible and could serve AAAA records. We added IPv6 addresses to our glue records with our registrar, so from the furthest point out that we could touch to the Internet, it would be able to browse IPv6 from that point. Once that was operational, we started pushing IPv6 to a select group of customer domains. We called a few customers asking if they wanted to be part of an experiment. A few said yes, so we enabled their domain on IPv6 and added some proper DNS records. What we noticed off the bat was that any email traffic headed to big name carriers, like Office 365 or Google, automatically went through IPv6. To us, it felt like those connections were faster.  I don’t know if they were, but I feel like there is less latency that carries through the entire connection with IPv6 due to less analysis of every individual packet like in IPv4. Everything seemed to go really well in the sample group we started with. So, we decided to go all in and enable it for every hosting domain that we had, unless someone requested otherwise.

IPv6 on the Internet service side

In the last decade, Watch has added terrestrial wireless Internet connectivity to customers, website and email hosting, fiber to home and business initiatives. We haven’t been able to push IPv6 out to those customers on a large scale yet. Our routers along our edge and along our access point all support IPv6 and we can push IPv6 out without any problem. We use PPPoE which makes pushing IPv6 out to customers very simple, but it seems like every individual vendor for home routers has their own way of handling IPv6.  In many cases,  customers would need to reboot all their home devices to get a new IPv6 address, and it is bad business to force a situation to create an inconvenience for a customer.  This is not the case with IPv4, since it’s all handled through NAT. We don’t want to force our customers to buy new gear, and until their router breaks, they are probably not going to go buy a new one. I suspect the solution is that ISPs like us are going to have to work with router vendors to sell or lease a router to every customer to get IPv6 pushed out to the customer edge.

We do have IPv6 rolled out to the customer edge in some limited areas. We made phone calls to a couple tech savvy customers who have purchased a new router and asked if they wanted to be guinea pigs. For those who said yes, we enabled IPv6 on those Points of Presence on the network similar to the hosting customers. They got their IPv6 prefix delegation. Those customers picked up. Everything worked.  More tech savvy users did notice some websites gave them some trouble. They could browse there yesterday, but today with IPv6, they couldn’t. We discovered a website our customer was trying to access had enabled IPv6, but it did not handle firewalls appropriately. IPv4 requests were coming in fine, but IPv6 requests were getting mangled which caused them to disconnect. We happened to know the operator of the site, so we worked with them to solve the problem. Luckily, we had this experience with a tech savvy user, so we got through it ok, and it was a good learning experience to help walk less tech savvy users through the same issues.IP Geolocation: The Good, The Bad, & The Frustrating 6

Push IPv6 further into the network piece by piece

We were prepared moving in to our IPv6 deployment and knew what to expect. Some old pieces of equipment weren’t going to be able to handle static IP routing, so we upgraded them.  We began by rolling things out at our network core where BGP comes into the network. We asked, can IPv6 operate there? If yes, then we went to the next set of routers upstream from that. We worked our way down the line one step at a time. Every couple of days, we pushed IPv6 further into the network. Everything went fine.

As expected, privacy extensions gave us trouble in our office locations, especially for some operating systems. They caused connectivity problems when we did software deployments, made updates to new programs, or remotely accessed computers to offer assistance. Sometimes DNS wouldn’t keep up fast enough with the new IP that the privacy extension was handing out, so we’d try to connect, and it wouldn’t work.  The problem turned out to be an old IP that was getting ready to expire from the privacy set and wasn’t taking new connections inbound. Disabling privacy extensions across the board ended up fixing that issue.

Don’t be intimidated

IPv6 can be extremely intimidating, especially for people who are not IP infrastructure engineers. They look at an IPv4 address, and it’s something small and familiar. They look at IPv6, and all of the sudden, it’s not just numbers, it’s a hexadecimal address, and there are letters. The advice we gave our staff was to not let that intimidate you. It is really just the difference of calling someone by their first name or calling someone by their first, middle, and last name. Don’t let it scare you. Once we got into teaching folks how our IP structure is planned out and how our addressing is being done, a lot of the staff actually ended up liking IPv6 better.  From our infrastructure it was easier for them to figure out where the IP was coming from. With IPv6 we started using the 4th nibble so that it was our building ID.  It just so happened that our building IDs fit perfectly with hex addresses. Helpdesk staff can look at an IP address and easily figure out the building and device whether it’s a printer, desk phone, or work station.

IPv6 & NAT

Many people seem to be against NATing in IPv6, but I think it still has a function in IPv6 with prefix delegations. We consulted with a few other businesses who were looking at deploying IPv6. They had an IPv6 address from their carrier, but were having the same issue we were having with customers. They didn’t have a route out to the Internet. The solution we came up with for that partner was that they would implement internal IPv6 NAT. Their internal IPv6 address was a unique local prefix that was a purposeful prefix. So they generated a prefix out of that range that they used for their internal network, and then it’s NATed before it gets to the Internet. Then whatever address they get from their carrier can change every hour but internally their devices don’t know. Their firewall does the translation and then out to the Internet they go. That solved that problem for them immediately and worked fine for two years. Everyone wants to do native public IPs all the way to the work station, and if your environment will allow it, and your firewall and your carrier can handle that appropriately, great. If it can’t, don’t dismiss NAT.IP Geolocation: The Good, The Bad, & The Frustrating 7

Take the time to learn how IPv6 works

Organizations will want to take a look at how IPv6 fits into their environment and works for them, not against them. We didn’t want to push our deployment so fast that we didn’t spend the time to do it right. Slow down. Take the time to brainstorm how you can make it work for you.  We took a step outside of the box and decided to look at what identifiers we could use to put into subnets to help tell what building or device an IP is assigned to. Some organizations may have multiple stories in an office building and the floor number comes into play. Take some time and really think about how IPv6 is going to be best implemented for you.


The post Make IPv6 Work For You, Not Against You appeared first on Team ARIN.

Posted on Leave a comment

A Common Mistake with DHCPv6

IP Geolocation: The Good, The Bad, & The Frustrating 8

A few weeks ago, I visited a former customer at their office. When I passed the coffee room, someone saw my IPv6 t-shirt and said, “Since you know everything about IPv6, why do we have a problem between server1 and server2? They are located on the same VLAN, and when we ping between them with IPv6 it’s unstable, but it’s 100% stable with IPv4.”

This organization has a few hundred Windows 2008 (or newer) servers in some VLANs where most of them have IPv6 enabled through Dynamic Host Protocol version 6 (DHCPv6), and very few have static IPv6 addresses. We sat down and tried a ping where the roundtrip time was unstable, and sometimes we also lost packets.

I suspected that the /64 prefix didn’t show where it was announced through the Router Advertisements (RA), and a Route -6 print confirmed that.

IP Geolocation: The Good, The Bad, & The Frustrating 9

Remember the Prefix Option in RA When Doing DHCPv6

This is a common mistake with DHCPv6. It isn’t just a mapping of DHCP for IPv4 functionality to IPv6. DHCPv6 does not provide the address, subnet mask, and default gateway that DHCP for IPv4 does. RAs provide the prefix and default gateway to the client, which then use that prefix to auto-config the interface on that network. Roughly put, functionality (DHCP for IPv4) = functionality (DHCPv6 + RA).

When we activated the RA with the correct /64 and enabled the on-link flag, the IPv6 between those servers worked perfectly.

Notice the route for the /64 prefix in the Route -6 print after:

IP Geolocation: The Good, The Bad, & The Frustrating 10

The customer enabled IPv6 more than five years ago in the same way, and though they may have had a problem with IPv6 earlier, this was the first time they discovered the problem.

In theory, we shouldn’t have this problem with this setup, but do you know anyone who has a perfect computer and network that works 100% according to standards?

IPv6 in an Enterprise Environment

When dealing with IPv6 in an enterprise environment, you must understand:

  • Managed and Other flag for DHCPv6
  • Autonomous and on-link flag

If you run DHCPv6 without Autonomous and on-link flag in a shared VLAN environment you can get in trouble.

You must also learn to configure the network equipment with the flags above. In this case, the syntax looked correct but when we pressed “?” after, it showed more parameters that should be included.

When to use DHCPv6 vs SLAAC

You may wonder why it is better to use DHCPv6 and not Stateless Address Auto-Configuration (SLAAC) in this case then? In an enterprise network where you must have 100% control, SLAAC isn’t enough in my opinion. With OptionIP Geolocation: The Good, The Bad, & The Frustrating 1118 or 37 you can get a persistent lease of just one IPv6 address/host/etc. per device.

With my 17 years of experience with IPv6, I know that the network administrators like when it’s “IPv4-like.” There is one exception when SLAAC is an option – use it when you have one VLAN per host in a large enterprise network.

In my defense, I had not helped the customer with this setup! Luckily, I was able to help them fix the problem all thanks to someone noticing my t-shirt, “A home without IPv6 is just a house.”

The post A Common Mistake with DHCPv6 appeared first on Team ARIN.

Posted on Leave a comment

A Beautiful Way to Run the Network

IP Geolocation: The Good, The Bad, & The Frustrating 12

Communicate Freely IPv6 Case Study

Communicate Freely in Port Perry, Ontario has 100% IPv6 availability to all customers.  As the founder of the company, I made an executive decision to deploy IPv6 because it’s the right thing to do. As the only shareholder at the time, I didn’t have to convince others with an immediate return on investment and was able to make the decision on its technical merits. We wanted to be part of the solution, not part of the problem. The business justification I’m using now that the business has grown, is that we are going to have do this to at some point, so why not do it now instead of scrambling at the last minute?  That justification works in nearly any organization – this is inevitable, so start making the changes sooner while the risk is lower.  Right now, if I break IPv6, no one notices. When IPv6 becomes mission critical, I know everything will work just fine since we will have been using it for years.

100% IPv6 availability

We are a small telecommunications service provider consisting of 10 employees, 250+ fiber to the home internet customers, and 800 PBX phone customers. Everything we do including our website, email, office, and Internet service is dual stack. All our core services are either dual stack or IPv6-only. Every Internet customer we provide service to can get IPv6.

When we provide residential fiber to the home, our customers get an optical network terminal (ONT) which is the device the fiber connects to. Most customers use that device as their router as well, so when we turn on IPv6 in the ONT, they have dual stack for their whole house. Some people will use their own router, and if it’s old, it may not support IPv6. I’d say at least 50% of our customers have taken a prefix delegation, and they are connected to our IPv6 network. Right now, our traffic is 20-25% IPv6, and I am trying to push that up.  Below is a graph from our router showing IPv4 vs IPv6 traffic.  It’s certainly not at parity yet, but we can see that IPv6 traffic is becoming significant.  On some days, I have seen IPv6 traffic exceed IPv4 traffic for brief periods.IP Geolocation: The Good, The Bad, & The Frustrating 13IP Geolocation: The Good, The Bad, & The Frustrating 14

Every now and then we see our traffic drop and I’ll wonder what went wrong. Maybe there’s a kink in our new DHCP server, or some other service isn’t functioning properly? Even if no one complains, I’ll fix whatever problem it might be. I have a laptop on our test bench plugged into a test ONT with IPv4 turned off. I can check the weather, play a YouTube video, etc. Every now and then I like to completely disable IPv4 and see what works and what doesn’t. With DHCP working correctly, the network assigns the DNS servers and all the things you need for the network to work.  Everything site that is IPv6-enabled works just fine.IP Geolocation: The Good, The Bad, & The Frustrating 15
I have identified 4 key pieces of our IPv6 readiness for ISPs:

Getting your allocation from ARIN

Believe it or not, we’ve had our IPv6 allocation longer than we’ve had IPv4. When ARIN was getting close to exhaustion, I had to show I was using at least a /23 to justify my IPv4 request, but I couldn’t find an upstream provider who had enough space to sub-assign to me.  The IPv4 address pool was already that constrained. I had customers ready to sign up, but our network only had an IPv6 allocation, so at first we NATTED everybody in IPv4 and gave them native IPv6.

Ensuring IPv6 support in the network equipment itself

A lot of equipment in our core supported IPv6 with no problems. Access equipment like our Fiber OLT mostly supported it, but customer equipment like ONTs, modems, routers and telephone end points would say they supported IPv6, but then when you went to use it, it wouldn’t work. It was missing a piece of functionality or didn’t configure properly. Only about a month ago could you finally create an IPv6 firewall rule in the latest firmware on our fiber ONTs.  You could connect to the IPv6 Internet, but you couldn’t allow any traffic to come in. If you wanted to run an email or web server there was no way to allow that traffic through the firewall inbound. They had all the port forwarding in IPv4, but did not have an IPv6 equivalent. Finally, in the last two months, equipment vendors have reached a level of maturity in the software on the customer end points where it has the equivalent functionally to IPv4.

IT staff learning how it all works

Honestly, IPv6 basics aren’t that hard. Figuring out how to do it at the service provider level was more of the challenge since there wasn’t a lot of information out there and we were the first ISP in our area to have IPv6 deployed. Figuring out DHCPv6 took me some time, but eventually I got it to work. It ended up being a lot easier than I thought it would be. I poked at it a lot and experimented. Eventually I found some documentation from someone who had done something similar enough that led in the right direction. What I really needed to see was a config file, and to know what do I actually set this to in my network? What really made everything work properly in IPv6 for me was when we deployed a new open source DHCPv6 server called Kea, by the Internet Service Consortium. I think the older DHCP version we were using had some bugs in its IPv6 implementation. The software was supposed to do it, but there were little bits of functionality not working right. The router has to do DHCP relay in order for that to get passed to the server and then it has to be aware of what allocations have been given to the customer so that it can forward traffic properly.  This is something that was not required for a typical IPv4 deployment, but with IPv6, the edge routers need to add routes for the address space assigned behind each customer router.

Having support for IPv6 in end point customer devices

The biggest barriers to IPv6 adoption we faced were vendor issues, especially customer premise equipment (CPE) like residential routers that couldn’t do IPv6. Or for example, in the ONT, IPv6 was turned off by default, so I had to go in and turn it on in all of them when we first installed it, and every time it reset. However, in a later release our vendor added some features where I could create a profile that will allow IPv6 to be on by default. Once that happened, and after the upgrade, we saw a significant increase in customers using IPv6, and a corresponding increase in traffic.

I think vendors need to step up. No product should be designed or sold that doesn’t support IPv6. If you’re a vendor or software developer you should be supporting IPv6 in all your products.IP Geolocation: The Good, The Bad, & The Frustrating 16

Future proof your network

My hope is that if we can have everything working well in IPv6, when we exhaust the /22 of IPv4 that we have, then maybe we won’t need to buy addresses on the secondary market.  In our lower tier packages, we can provide an IPv6-only connection, and use a NAT64 translator to reach the remaining parts of the IPv4 Internet.  Things will still work fine for most users, and we can look at IPv4 as an optional service for businesses or specialty users that need to support the older technology. IPv6 is future proofing our network. It will save us some long-term costs, and keep our network on the leading edge.

Benefits of using IPv6

As a network engineer, I’d love if we could get to the point where I could just not deal with IPv4 anymore. Private address space is pain, NAT is a pain, and I’d rather just use IPv6. I’m slowly reducing what I put on private IPv4 address space and use IPv6 only instead.

We have database servers and internal websites that don’t even have an IPv4 address. We just don’t have to manage that. If we need to allow remote access for people outside the network, we have firewall rule verses tunneling and NAT. We just let people in with a simple rule set as opposed to a more complicated solution.

I am OCD when it comes to numbers, and IPv6 is a lot easier to manage than IPv4. For example, if you are using private address space you have to make sure they don’t overlap with other networks you may connect to, or within your own network. Because we have to turn on IPv6 anyway, why not just do the one instead of both? It’s easier to manage one protocol than two. I think IPv6 is easier to work with because the subnets are bigger and everything is unique. If I assign a /64 to a subnet, I know I will never outgrow it. Whereas with IPv4, I have to always make it just big enough, because we are limited to the number of addresses we have.  Another bonus with IPv6 is that everything is on the same boundary verses in IPv4 where you have to have subnet boundaries in the middle of blocks. I can take whatever byte position that is and increment it so I can make that line up with VLANs or other non IP things that are in the network. You can be a lot more orderly with your address management, because you are not tied to a constrained address space.

At the end of the day, all the things that are working on IPv6 save me time. I believe the customer experience will be improved in the long run as well. When everyone runs out of IPv4, we won’t be translating. In some cases, latency is lower on IPv6 so content loads faster. This is a performance benefit to using IPv6 that is small right now but will increase.
IP Geolocation: The Good, The Bad, & The Frustrating 17

IPv6 is a very elegant protocol and easier than everyone thinks it is. Do it because it’s a beautiful way to run the network and don’t be afraid of it. If everyone would just do IPv6, every network administrator’s life would be so much easier, and the longer we put it off, the more ugly the Internet will look in the meantime.

The post A Beautiful Way to Run the Network appeared first on Team ARIN.

Posted on Leave a comment

ARIN Bits: June 2018

We’re diving into summer with a productive second quarter behind us! In this edition of ARIN Bits, you’ll learn about our new fee schedule, the ARIN fellowship program, key election dates, our 2017 annual report, tips from our Registration and Financial Services teams, and more.  Did you miss last quarter’s edition? You can find all past editions on our ARIN Bits page.

New Fee Schedule Effective 1 July

At its 24 May 2018 meeting, the ARIN Board of Trustees adopted a new fee schedule effective 1 July 2018. The new fee schedule reflects proposed modifications and community feedback collected from both the ARIN 41 Public Policy and Members Meeting as well as the Community Consultation that concluded on 25 May 2018. New fees will be reflected in invoices due on and after 1 July 2018. Have a question? Take a look at our Fee Schedule FAQ page.

Join us in Vancouver as a fellow!

Our next Public Policy and Members Meeting (PPMM), ARIN 42, will be held in Vancouver, BC from 4-5 October 2018 at the Hyatt Regency Vancouver. Registration will open in the coming weeks!

Applications are currently being accepted for our Fellowship Program to attend ARIN 42 with the option to attend NANOG 74 from 1-3 October. Apply now through 5:00 PM EDT on Wednesday, 11 July 2018. Not sure what a fellowship entails? Take a look at our Fellowship Corner to meet our former fellows and hear about their experiences first hand.

Nominations for ARIN Elections Open 2 July

Beginning 2 July, ARIN Trustees and representatives from our General Members in Good Standing are cordially invited to nominate candidates for seats on the Board of Trustees, Advisory Council, and Number Resource Organization Number Council (NRO NC) to serve three-year terms beginning 1 January 2019.

To view the initial requirements and responsibilities of the Board of Trustees and/or Advisory Council, please visit the pages below:

  • Board of Trustees
  • Advisory Council

Also, remember to update or establish your Voting Contact by 20 August! Be sure to mark all key election dates this year.

July Outreach Events

We’ll be speaking about IPv6 at the EDCO New Technology Briefing in Montreal on 14 July. Attend this free program sponsored by Enterprise Data Center Operators (EDCO).

We’ll also have a booth at the CANTO 34th Annual Conference & Trade Exhibition in Panama City, Panama 22-25 July. Come visit us and let’s talk about how we can provide you Internet number resources and other services!

Our 2017 ARIN Annual Report is now available!

The report includes:

  • An overview of our mission, services, and structure
  • Updates from our President & Chief Executive Officer and Board of Trustees Chairman
  • A summary of 2017 accomplishments from our Chief Operations Officer and Service Level Report
  • Department highlights and Internet Governance participation report
  • A recap of 2017 outreach events and public policy discussions
  • An overview of our Policy Development Process (PDP) and Number Resource Policy Manual (NRPM) changes
  • Activity reports from the Board of Trustees, Advisory Council, and NRO Number Council
  • Internet number resource statistics for 2017 and historical activity

View the full annual report. The 2017 Auditor’s Report is also available.

We have a few policy proposals under discussion, including:

Pending Board of Trustees Review:

  • ARIN-2017-3: Update to NRPM 3.6: Annual Whois POC Validation
  • ARIN-2017-8: Amend Community Networks
  • ARIN-2017-10: Repeal of Immediate Need for IPv4 Address Space (NRPM Section
  • ARIN-2017-12: Require New POC Validation Upon Reassignment
  • ARIN-2017-13: Remove ARIN Review Requirements for Large IPv4 Reassignments/ Reallocations

Recommended Draft Policies:

  • ARIN-2018-1: Allow Inter-regional ASN Transfers

Draft Policies:

  • ARIN-2018-2: Clarification to ISP Initial Allocation and Permit Renumbering
  • ARIN-2018-3: Remove Reallocation Requirements for Residential Market Assignments
  • ARIN-2018-4: Clarification on IPv6 Sub-Assignments

Editorial changes under discussion:

  • Reallocation and Reassignment Language Cleanup (formerly Draft Policy ARIN-2017-11)
  • Correct References to RWhois (formerly ARIN-prop-251)

You can find the status of current policy discussions and subscribe to ARIN-PPML (Public Policy Mailing List) to voice your opinions. And remember, membership is not required to participate!

In May we added some new features to ARIN Online:

  • We completed a number of infrastructure improvements and performed multiple bug fixes.
  • Pages and processes in ARIN Online were improved and streamlined, including:
    • Separate IP address request flows were created for ISPs and end users.
    • On the View & Manage Network page, the Parent Org name now appears in the Parent field next to the network link.
    • The pages to recover Point of Contact records and Org IDs were improved.
    • The Organization POC Info page was revised, and an option to Manage Organization POCs was added to the Actions menu on the Organization Record page.
    • The Request ASN, Organization Record, Request Membership, and Edit Org Name pages were revised.
    • ASNs in the ASN Number to Name file at are now listed individually, instead of in ranges. (ACSP 2017.18)
  • Fee invoice attachments have been revised to include how to obtain additional information, such as a list of resources covered under the invoice. (ACSP 2017.5)

Our Featured Policy Requirement:

Did you know that under NRPM 8.5.7: Alternative Additional IPv4 Address Block Criteria, if you are utilizing at least 80% of your currently allocated/assigned IPv4 addresses, you may qualify to receive one or more transfers of IPv4 addresses equal to your current holding (maximum of a /16 equivalent in any six-month period)?

A Tip from Our Registration Services Department:

Do you have an old reassignment/reallocation from an ISP you are no longer working with and would like it removed? Please review the documented process outlined on our website for the Removal of Stale Reassignment / Reallocation Records.

A Tip from Our Financial Services Department:

Transfer Processing Fee (TPF): Invoice and Payment Facts:

  1. Each transfer request will be invoiced a $300 US fee, billed to the source (or legal successor) organization
  2. Payment of the TPF is required before the review process can begin
  3. The payment is non-refundable and does not guarantee approval of a transfer request
  4. The TPF is waived when the subject resources are under an existing Registration Services Plan (RSP), and no specific transfer processing fee will be charged to the recipient-side organization for 8.3 and 8.4 transfers.
  5. For each transfer approved, the recipient organization:

-Must have an updated and executed Registration Services Agreement

-May be subject to an initial fee(s)

Transferred resources are also subject to annual fees as stipulated by the Fee Schedule.

Check out these Customer and Member Statistics (As of 31 May 2018)

  • 5,783 member organizations
  • 758 8.3 transfers and 67 8.4 transfers completed YTD 2018
  • 8.4 transfers completed YTD 2018: 31 to APNIC, 5 from APNIC, 26 to RIPE NCC, 4 from RIPE NCC
  • 57.6% of members have an IPv6 block

Here are a few helpful IPv6 links:

  • Dedicated IPv4 block to facilitate IPv6 Deployment – under this policy, ARIN set aside a contiguous /10 IPv4 block for allocations and assignments justified by immediate IPv6 deployment requirements. This block is subject to a minimum size allocation of /28 and a maximum size allocation of /24.
  • List of IPv6 Training Consultants
  • Software Developers Guide to Writing and Migrating Network Applications for Use on IPv6 Networks
  • IPv6 Number Planning

Catch up on our recent posts:

  • ARIN Website Redesign – Phase 1 in Review 
  • IPv6 Case Studies
  • Tips from Registration Services Department
  • TipsFor6 Campaign
  • ICANN’s Updated KSK Roll Operations Implementation Plan

We’ll see you next quarter, have a great summer!

The post ARIN Bits: June 2018 appeared first on Team ARIN.

Posted on Leave a comment

ARIN Website Redesign – Phase 1 in Review

Redesigning any website is no small feat, and ARIN’s website is no exception.  The ramp up to our Phase 1 preview at ARIN 41 included a full site audit, creation of the ARIN Vault, card sorting exercises involving staff and community, and paper prototyping with volunteers at ARIN 40. We formed a special cross-department team involving communications, operations, software development and user experience. It has been a wild ride so far.

Next up will be testing options for ARIN Online integration with the new website.

But back to the preview, when we returned from ARIN 41 we spent some time reviewing the feedback we received online and in-person, as well as through our online tree test of our proposed navigation hierarchy.  We received lots of great input that we have already put to use in shaping the next phases of our redesign effort.

Some Numbers

  • The Preview homepage received approximately 2500 total pageviews and 700 unique page views
  • Eleven ARIN 41 attendees sat for in-person user tests that lasted an average of 11 minutes
  • Thirty-eight individuals completed the feedback survey available through the preview site
  • One hundred twenty-four individuals started the navigation Tree Test; 72 completed the study, 52 abandoned – 14 of 27 questions were answered on average
  • Over 70% of respondents said they found the main menu either somewhat or very intuitive. Sub-categories got a similarly favorable rating with 52% saying they were very or extremely helpful. An additional 44% found them somewhat helpful.
  • When asked about the five “top task” items on the homepage, 65% of respondents found them to be extremely or very helpful. The main complaint was that they were not necessarily helpful to power users/frequent visitors.

Top Take Aways

 In-person User Tests


In-person testers gave generally high marks to look and feel

  • Common descriptors were: clean, easy, organized
  • People like videos


  • A couple of testers at the Miami event, not unexpectedly, suggested Events could be higher on the home page
  • A few testers were resistant to scrolling

Preview Feedback Survey 

  • We received 19 freeform comments on the navigation menu, including several items that have already been identified as stories for improvements:
    • Shortening the delay on the menu hover
    • Ordering and number of topics in the sub-menu
    • Changes to headings
  • Some actions under consideration:
    • Adding a Username and Password field for account login
    • Using a single color for announcements
    • Improvements to the top task graphics and labeling to make them more intuitive
    • Reorganizing the lower half of the page to reduce “clutter”
    • Updates to the top-level navigation and sub-menu
    • Improvements to the search box to make it more intuitive to use

Tree Test

  • Naming conventions, especially as applicable to page titles, should be reconsidered for clarity
  • Top-level navigation needs some modification, possible additions
  • Some issues in the test are resolved by the mega-menu and it’s presentation of related options in a single drop-down. We could not test exactly as the menu displays
  • Need landing pages for ASN and IPv4
  • IPv6 content should be consolidated
  • Users currently expect to find POC info under Manage Resources
  • Users currently expect to find Org Id info under Getting Started
  • Tester had no clear idea of where Membership info should be located
  • Expected Elections info under Participate not About

This is just a sampling of the kinds of great insights we received from our preview testers, and there will be more opportunities for the community to give feedback before we launch our new website!

Coming Soon

ARIN Online and integration options – help us shape your path from the website to the customer account application.

ARIN Website Preview Testing at ARIN 42 – Last looks! This will be the final opportunity for the community to provide feedback on the new website prior to launch in 2019!

The post ARIN Website Redesign – Phase 1 in Review appeared first on Team ARIN.