• At Kemp Little, we are known for our ability to serve the very particular needs of a large but diverse technology client base. Our hands-on industry know-how makes us a good fit with many of the world's biggest technology and digital media businesses, yet means we are equally relevant to companies with a technology bias, in sectors such as professional services, financial services, retail, travel and healthcare.
  • Kemp Little specialises in the technology and digital media sectors and provides a range of legal services that are crucial to fast-moving, innovative businesses.Our blend of sector awareness, technical excellence and responsiveness, means we are regularly ranked as a leading firm by directories such as Legal 500, Chambers and PLC Which Lawyer. Our practice areas cover a wide range of legal issues and advice.
  • Our Commercial Technology team has established itself as one of the strongest in the UK. We are ranked in Legal 500, Chambers & Partners and PLC Which Lawyer, with four of our partners recommended.
  • Our team provides practical and commercial advice founded on years of experience and technical know-how to technology and digital media companies that need to be alert to the rules and regulations of competition law.
  • Our Corporate Practice has a reputation for delivering sound legal advice, backed up with extensive industry experience and credentials, to get the best results from technology and digital media transactions.
  • In the fast-changing world of employment law our clients need practical, commercial and cost-effective advice. They get this from our team of employment law professionals.
  • Our team of leading IP advisors deliver cost-effective, strategic and commercial advice to ensure that your IP assets are protected and leveraged to add real value to your business.
  • Our litigation practice advises on all aspects of dispute resolution, with a particular focus on ownership, exploitation and infringement of intellectual property rights and commercial disputes in the technology sector.
  • We have an industry-leading reputation for our outsourcing expertise. Our professionals deliver credible legal advice to providers and acquirers of IT and business process outsourcing (BPO) services.
  • We work alongside companies, many with disruptive technologies, that seek funding, as well as with the venture capital firms, institutional investors and corporate ventures that want to invest in exciting business opportunities.
  • Our regulatory specialists work alongside Kemp Little’s corporate and commercial professionals to help meet their compliance obligations.
  • With a service that is commercial and responsive to our clients’ needs, you will find our tax advice easy to understand, cost-effective and geared towards maximising your tax benefits.
  • At Kemp Little, we advise clients in diverse sectors where technology is fundamental to the ongoing success of their businesses.They include companies that provide technology as a service and businesses where the use of technology is key to their business model, enabling them to bring their product or service to market.
  • We bring our commercial understanding of digital business models, our legal expertise and our reputation for delivering high quality, cost-effective services to this dynamic sector.
  • Acting for market leaders and market changers within the media industry, we combine in-depth knowledge of the structural technology that underpins content delivery and the impact of digitisation on the rights of producers and consumers.
  • We understand the risks facing this sector and work with our clients to conquer those challenges. Testimony to our success is the continued growth in our team of professionals and the clients we serve.
  • We advise at the forefront of the technological intersection between life sciences and healthcare. We advise leading technology and data analytics providers, healthcare institutions as well as manufacturers of medical devices, pharmaceuticals and biotechnological products.
  • For clients operating in the online sector, our teams are structured to meet their commercial, financing, M&A, competition and regulatory, employment and intellectual property legal needs.
  • Our focus on technology makes us especially well positioned to give advice on the legal aspects of digital marketing. We advise on high-profile, multi-channel, cross-border cases and on highly complex campaigns.
  • The mobile and telecoms sector is fast changing and hugely dependent on technology advances. We help mobile and wireless and fixed telecoms clients to tackle the legal challenges that this evolving sector presents.
  • Whether ERP, Linux or Windows; software or infrastructure as a service in the cloud, in a virtualised environment, or as a mobile or service-oriented architecture, we have the experience to resolve legal issues across the spectrum of commercial computer platforms.
  • Our clients trust us to apply our solutions and know-how to help them make the best use of technology in structuring deals, mitigating key risks to their businesses and in achieving their commercial objectives.
  • We have extensive experience of advising customers and suppliers in the retail sector on technology development, licensing and supply projects, and in advising on all aspects of procurement and online operations.
  • Our legal professionals work alongside social media providers and users in relation to the commercial, privacy, data, advertising, intellectual property, employment and corporate issues that arise in this dynamic sector.
  • Our years of working alongside diverse software clients have given us an in-depth understanding of the dynamics of the software marketplace, market practice and alternative negotiating strategies.
  • Working with direct providers of travel services, including aggregators, facilitators and suppliers of transport and technology, our team has developed a unique specialist knowledge of the sector
  • Your life as an entrepreneur is full of daily challenges as you seek to grow your business. One of the key strengths of our firm is that we understand these challenges.
  • Kemp Little is trusted by some of the world’s leading luxury brands and some of the most innovative e-commerce retailers changing the face of the industry.
  • HR Bytes is an exclusive, comprehensive, online service that will provide you with a wide range of practical, insightful and current employment law information. HR Bytes members get priority booking for events, key insight and a range of employment materials for free.
  • FlightDeck is our portal designed especially with start-up and emerging technology businesses in mind to help you get your business up and running in the right way. We provide a free pack of all the things no-one tells you and things they don’t give away to get you started.

Amazon and the $150 million typo: cloud risks for early stage companies, and how to mitigate them

Although the impact was not quite as big as some headlines had suggested (“Amazon Just Broke the Internet”), the outage of Amazon’s Simple Storage Solution (S3) in the US-East-1 region on Tuesday 28 February caused significant disruption. The Wall Street Journal quoted Cyence Inc., a start-up specialising in cyber-risks, as estimating that the Amazon outage cost companies in the S&P 500 index $150 million. Apica Inc., a website-monitoring company, said 54 of the internet's top 100 retailers saw website performance slow by 20% or more. Connected lightbulbs, thermostats and other IoT hardware were also impacted, with many unable to control their devices as a result of the outage. Nest warned customers that its internet-connected security cameras and smartphone apps were not functioning properly as a result of the Amazon issue. Amazon was unable to update its own Amazon Web Services (AWS) status dashboard for the first two hours of the outage because the dashboard itself depended on the unavailable systems.

Amazon’s explanation was that “an authorized S3 team member using an established playbook executed a command which was intended to remove a small number of servers for one of the S3 subsystems that is used by the S3 billing process. Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended." Removing a significant portion of the server capacity required full restarts and this problem was compounded by the fact that parts of the system had not been completely restarted for several years, a process which took longer than expected.

As a result of the outage, Amazon said it is making several changes to the way its systems are managed and promised to make changes to improve the recovery time of key subsystems.

In signing up to cloud hosting contracts, a lot of companies assume everything will be fine and their websites, applications and data will always be available when needed, particularly if they are choosing one of the leading providers of hosted services such as AWS. In August 2016 Gartner identified AWS and Microsoft as the only two companies in its “Leader” category for cloud infrastructure as a service (IaaS) worldwide (ranking AWS ahead of Microsoft) and said that “The market for cloud IaaS has consolidated significantly around two leading service providers.”. This consolidation increases the impact of outages such as the one impacting Amazon’s S3 service.

Given the potential impact of an outage on critical services, customers may need to reconsider how they mitigate the risk of downtime, and we discuss the possible options below.

Increasing the target for availability

Taking Amazon’s S3 service as an example, when used in a single region it is said to be designed for 99.99% of availability with a service level agreement for availability of 99.9%. However, relying on a service in a single region offers the potential for a single point of failure. The Amazon outage on 28 February involved just one region, US-East-1 in northern Virginia USA, but the impact of the outage was so significant as this is the most heavily-used regions in the AWS global infrastructure.

The impact would not have been so significant if AWS customers had chosen a multi-region architecture as sites and applications using S3 in a different region would not have been affected. AWS currently operates 42 availability zones (AZs) within 16 geographic regions around the world. AZs consist of one or more discrete data centers, each with redundant power, networking and connectivity, housed in separate facilities, miles apart from each other on separate flood plains. By contrast, another of AWS’s services, EC2, provides an SLA of 99.95% but this greater availability threshold is based on deployment to at least two AZs (although S3 can only be selected by region, not by AZ).

The disadvantage of this approach is that multi-region implementations will increase cost and complexity. Customers are understandably reluctant to achieve an extra ‘9’ of availability by selecting another region and potentially doubling their hosting costs. However, the additional costs and complexity will need to be measured against the risks of operational disruption, financial loss and reputational damage arising from significant unavailability of critical data and/or applications in a worst case scenario.

Negotiating a stronger contractual position

Contracts with major hosting providers usually restrict the customer’s remedy to service credits if the provider fails to meet its availability target. For Amazon’s S3 service for example, if availability falls below the service level of 99.9% in a month customers would typically be awarded a service credit of 10% of the monthly fee. This may well be wholly insufficient recompense to customers who need to ensure that they can access their data or keep their sites and applications up and running at critical times, particularly if the service credits do not cover customers’ liability to their own customers as a result of unavailability.

The major hosting providers have shown some willingness to offer more contractual protection for their customers by offering increased limits on their liability for damages caused by service level failures but this has come with a significant cost in terms of fees or only been available to customers spending very significant sums with the hosting provider. Such additional legal protection has typically not been afforded to customers spending less, and this is understandable: from the hosting providers’ perspective, they are offering a low-cost and largely commoditised solution and it is simply not realistic to expect them to carry significant legal risks at the price point at which the lower end services are offered. In other words, you don’t get what you don’t pay for, and so at the cheaper end of the market where commoditised services are being provided, customers are very unlikely to be able to negotiate better legal protections.

However, where high levels of availability are essential to their business model customers should insist on having visibility over who is hosting their data and applications and ensure that during contract negotiations suppliers are required to identify all key subcontractors (and their subcontractors) so that the customer can identify potential vulnerabilities in the supply chain and consider steps to mitigate the risk of downtime before becoming committed to the contract.

Taking more control over hosting arrangements

Moving away from a massive scale, multi-tenant model towards a single-tenant, private cloud or even on premise deployment provides an opportunity for more control but at a cost both financially and in terms of operational flexibility. The cost benefits of deploying to the cloud are a significant source of advantage for start-ups and smaller organisations which do not have a major investment in existing on premise hardware, combined with the agility and flexibility of cloud computing and instant access to global infrastructure. In contrast, large enterprises deploying to the cloud face a considerable incremental cost in addition to maintaining legacy on premise resources until these can be retired, a process which may take several years.

Even for start-ups though, the need to take control over how critical services are delivered may outweigh the costs. Digital challenger bank Monzo, which offers a contactless prepaid Mastercard and plans to offer a free current account this year, said that a severe outage resulting in its cards and app not working for most of Sunday 5 March was caused by a third party processor used by Monzo to connect to payment networks. When it first started it made sense for Monzo to use a third party processor because the process for connecting directly to the payment networks was long, costly and complex and at the time there seemed to be no benefit to its customers. However, Monzo has just finished a 12-month project to connect directly to Mastercard so that it can process transactions entirely using its own technology. Announcing this change in a blog post published on 6 March, Monzo's head of engineering Oliver Beatties said that "We see ourselves as a technology company as much as a bank, and going forward our strategy is to bring all critical systems in-house and continue to develop our own platform atop modern technology which we control.”.

Local back-ups as a safety net

Despite the attractiveness of short-term savings in moving data and applications to a single region, cloud-based solution, this approach could end up being very costly if businesses are dependent on a single point of failure without having an alternative solution which they can access quickly. From a practical perspective, whatever model they adopt for hosted services customers need to ensure that they make regular back-up copies of their data stored by a hosting provider, downloading copies of the data to their own systems or to an alternative hosting provider so that if absolutely necessary they can quickly implement an alternative solution.

The same applies to software: keeping full back-up copies of key applications on-site means that, should a hosting provider have an extended outage, there is at least an option to redeploy elsewhere rather than risk an indefinite interruption in service.

Worth paying the extra hosting fees?

While cloud storage and processing does offer significant price and operational advantages for start-ups, it may well be worth even for early stage start-ups thinking about the relative costs of paying for hosting in an extra region and / or with alternative provider, relative to the impact on operational stability, reputation and customer retention that a prolonged full outage might have on a growing business. Even the most heavily negotiated hosting contracts are highly unlikely to afford adequate recompense for the effects of a full outage after it has happened. As such, while it is still strongly advisable to review the contracts (not least to ensure compliance with, for instance, data protection legislation), the strongest way to deal with the risks emanating from an outage is probably still to use an architecture for the hosting of data and software that will minimise the risk of there being a full outage in the first place.

 

 

Contact our experts for further advice

Stephen McGinley, Chris Hill