Enabling distributed digital business with API-first architecture (Google Cloud Next '17)

Just another WordPress site

Enabling distributed digital business with API-first architecture (Google Cloud Next '17)

ED ANUFF: So I’m going to talk to you today So first let me introduce myself I’m Ed Anuff, head of product strategy at Apigee at Google And what I’m going to talk to you today about is this idea of API first thinking, how you should think about your APIs, those digital products, some of the different ways that APIs get shared and reused And how you need to think about APIs as part of an ecosystem rather than just simply as a way that you are exposing your applications as digital services So, one of these– then there’s me, let me skip past that– generally when we talk about API first thinking, what we mean by that is this idea that the starting point of your APIs shouldn’t simply be what you have in the database, what you have in your application server, or in an enterprise context, what you have in your enterprise service bus But really starting from this idea of, what are the experiences that you’re trying to power? And what we see in sort of traditional IT and also I think from sort of a first order thinking that most developers who approach this problem come at is to say, I have this stack I have this application that I built I’ve got perhaps a legacy system that I want to make available And so APIs are just simply a way that I expose this service And depending on how I’m going about thinking about my API program, perhaps maybe I’m assembling an API team or maybe this is more of an informal micro service context and I’m just sort of sharing in an ad hoc way, these are the services I have available But generally, there’s this idea of what at Apigee we would call this digital value chain Which is that you’re exposing these services and a variety of participants, developers go and use these APIs and build applications And they ultimately may show up powering a set of applications that a user actually interacts with And API first thinking basically reverses that And it goes and says, OK, there are a set of experiences that I need to deliver There’s a set of partnerships that I want to build Perhaps I want to create new types of business models that are made possible by APIs And I need to start with that end in mind and design my APIs to meet those needs I need to think about what the developer experience is going to be I need to think about what the terms of service are going to be And I’m going to craft an API program around that And most people that we see start from that previous slide But the really successful API programs, the companies that you see that are actually based on APIs, so we all see companies like Twilio and Stripe that have been very successful in building API based businesses, they start in this way They start thinking about the developer experience and they start thinking about as much as possible, the user experiences that these APIs may ultimately power So, given that that’s the case, why don’t we do? This why doesn’t everybody do this? Well, the answer is there’s a lot of different types of APIs In fact, there are really three different ways of viewing APIs And one of the things that I hope you take away from this presentation is whenever the API discussion starts up in your development teams within your company, in whatever context people start talking about we should pursue an API strategy, start to ask some questions about what do we mean by these APIs? Because there’s at least three different forms of APIs There’s APIs really from a service perspective, whether we’re talking about new style micro services or old style service oriented architecture There’s APIs as interactions And then there’s APIs as products And we’ll look at each one of these in this presentation And all of these APIs exist within ecosystems That is the fundamental truth of an API is that it is designed to share services, share data, connect experiences between systems, between applications, between people So you have to think about the ecosystem consideration

regardless of what the purpose is, which of these three types of APIs that you’re building So, APIs as services, if you’re a developer, if you’re within IT or whether you’re in a startup, this is the area of APIs that I think most developers sort of intuitively understand because it’s what we live in, what we– it’s how we build our software these days But these APIs take a lot of different forms, even within this context The main way that most developers end up interacting with APIs is in order to get to the back end resources that their applications need And what we see is that if you’re building an application, you need access to data, you need to be able to interact with the outside world perhaps via email services Most people are not going and using their company wide email server to send out their email interactions They may be connecting to an enterprise service bus or some form of integration architecture if they’re part of an enterprise Those of you in the platform as a service space, if you’re using Cloud Foundry, if you’re using Heroku, this idea of the 12 factor model is going to be very familiar to you and this idea that your application workloads can scale better if you separate out the stateful services and treat them as external services So this might be a way that you come to start thinking about APIs because this is how you’re going to get your data or how you’re going to get it your storage or perhaps even to an external service And obviously, there’s a whole set of companies, Google included– this is the nature of the Google Cloud Platform– that are providing you with a set of capabilities that are going to be used to power your applications Interestingly enough, in the enterprise space– and some of you are in enterprise space, some of you are in the startup space– we see a big divide In the enterprise space, most of these services that we talk about are probably sitting behind your firewall, perhaps in your own data center Whereas if you are doing Greenfield new development if you’re a startup, hopefully you are getting these from the variety of awesome databases of service or storage service choices that you have But even if you’re doing this internally, you’re still thinking this way We see very few companies that are going and approaching building new internal services for their applications where they’re not now thinking about saying, how do we make these RESTful services, how do we make these API based? And in fact, increasingly going and using the popular developer portal type of format of allowing internal developers to discover and use these APIs And there’s a lot of interesting work that’s being done here to replace this ESV approach In the Cloud Foundry world and the Kubernetes world, there’s this concept of a service broker that provides a better level of mechanism for integrating these services Now inevitably in these conversations, we get questions about micro services And in fact, at Apigee as we interact with customers, this becomes the most common topic these days Which is wait a minute, we’re talking about doing micro services What’s the difference between APIs and micro services? And also by extension, what’s the difference between micro services service oriented architecture? So there’s a lot of definitions of this, of micro services that float around The one that I would have really stuck to when people would ask me what the difference between micro services and so is, is to really talk about the fact that micro services are trying to solve a scalability problem They’re trying to solve the challenge of decomposing your application into independent units of work that can be scaled at appropriate levels, the right container instances for those workloads Each one can be elastically scaled independently Micro services generally are not meant to be a mechanism of reuse Again, the best analogy to micro services are, we took the components of the spring world and now we’re laying these out into containers It lets us do things like have polyglot programming within our development teams These are all good reasons to use micro services Not to say that you can’t do micro services for the purpose of sharing and reuse, but you need to understand you are going to pay a penalty in terms of agility the wider you cast a net of your service dependency

graph The more people who depend on your services, the more you will translate from being– or transform from being a very agile application development team, which may have been the way you started this journey, to being a service provider who now has to worry about a whole bunch of other considerations And it becomes a full time job in and of itself These are the ecosystem considerations I was referring to previously, which I’m going to get into a little bit more depth later in this presentation So the next form of APIs, and this really throws a lot of people for a loop, is this idea of APIs as interactions And in fact, as we’ve worked with enterprises, and most enterprises really had embraced this idea of service oriented architecture in the early 2000s, and there were a set of blueprints and patterns for reuse that designing services– the level of service granularity and so forth– really were encouraged as best practices Those proved to be anti-patterns when you went to move towards mobile And they actually further hampered a lot of people as they tried to embrace things like IoT And there were a number of reasons for this But the idea was that these services that you use to power front end interactions look very different than services that you use to expose back end resources They are purpose built These experienced APIs– that’s a term that Netflix coined to describe these and it’s caught on for people who sort of study different API design patterns– look very different than the services that are designed for wide reuse that perhaps you’re using when you’re building a back end service that you want to be used by many different applications And in fact, the lifespan of these is very different In the mobile world, you further compound this with the fact that many mobile developers, if there’s no SDK to access these APIs, consider the API incomplete And so that’s why when you see companies that are trying to succeed with mobile developers, if you look at– for example, Firebase provides a real first class mobile experience They do so by providing very well-built SDKs that developers can easily integrate into whatever IDE environment that they’re using, Xcode, what have you They’ve put a lot of thought into this developer experience Most API developers, most API architects that I talk to, they sort of want to push as much as possible, this SDK problem off to the side You do see some good approaches A lot of the popularity of Swagger now, open API, has come about because it allows you to express these APIs a way that can quickly be accessed via SDKs, client libraries that can be auto generated from the APIs Now experience APIs add a new wrinkle in that some of these again are purpose built They are not designed for reuse They may be built by your front end developers So you may have one team that’s building your APIs for general reuse but you’ve got your web developers who are building single page apps who need special APIs to power those experiences And you may have the aforementioned mobile clients that again, needs specific APIs to power specific experiences Anybody who goes to a shopping site, you go to Amazon, you see a Buy It Now button Obviously downstream, you are going to go and have an inner API that handles the payment processing but there’s going to be a separate interaction or experience API that’s powering that Buy It Now button This is not an anti-pattern You can have multiple forms of APIs to power multiple forms of interactions The way you do that is by having this mediation layer And so, companies that are building user experiences around APIs have some form of mediation layer That might be something that was built in Node.js In fact, a lot of the popularity for Node has come from the fact that because it’s JavaScript, because JSON is the native data format, it’s a great place to go and basically create this mediation tier that can go and take this back end inner API that might’ve been designed for general purpose use and create these small services that are purpose built for perhaps a specific interaction when user hits a button on your mobile app The role of API management platforms, if you want to know why do a lot of companies deploy technologies like Apigee? API management platforms provide this mediation layer type mechanism And so, this might be a reason why you find yourself looking at that sort of product or technology

So the third aspect of APIs is API as products And this tends to really separate out the thinking from those two previous instances In the scenarios where you’re thinking about APIs as services, it’s really your developers, your software development team leads that perhaps are thinking about what services are being designed for wider reuse In APIs as interactions, it’s probably a front end team But now you’ve got the scenario where you go and say I want to build a business model around APIs And in that case, what you’re doing is you’re really thinking about how you are going to manage these APIs as products And this tends to trip people up because there’s a number of different folks involved in building APIs You’ve got your developers that build the APIs You have a set of people who are testing them You have a whole set of operational concerns, whether you’re doing sort of classic enterprise style centralized ops or whether you’re using more of a DevOps focus, you still have a whole set of considerations around that Now you’ve got to think about how you manage this API lifecycle And what we’ve seen is that companies that are successful with this have somebody in the role of API product manager And it’s not always a full time job It might be part of another role But there is somebody whose job it is to make sure that all aspects of the API lifecycle and most importantly, the critical metrics that determine whether the API is succeeding, are being looked after So when you’re productizing an API, suddenly you now have considerations and if you’re this API product manager, you’re very focused on your API design If you’re building a payment processing API– we’ve got a number of customers that are offering payment transaction gateways If you’re the product manager of one of those APIs, you’re going and saying, how can I provide a better developer experience than someone like Stripe? And the way you’re doing that is through superior documentation, perhaps better client SDKs, and you’re looking at the design of the API itself Is the amount of parameters I have to send or these payloads, are they the right size? Are they easy for developers to wrap their heads around? Or do I have to consider and send a lot of data that really doesn’t make sense for my application That’s where API design comes in Is it properly designed from RESTful best practices? A lot of new security concerns One of the things that we’ve seen as we’ve been in a lot of these API hackathons is that the biggest challenge people have, developers in using a new API is getting the authentication right It’s a big problem Simply just going and saying it’s OAuth isn’t always the solution There’s still a lot of things that need to be improved in getting developers to be able to understand and be able to go and make a cURL call that allows them to actually get results from the API And then the last piece, and this is probably the biggest set of considerations that the would be API product manager needs to think about, is the metrics for success This is the difference between APIs as a product and just APIs as a project Your metrics for success from an API are going to be different, depending on your business John Musser, the founder of ProgrammableWeb, several years ago made the observation that I agree with, that you need to– regardless of the context of what your APIs are for, you need to go and think about the classic marketing funnel Some of you are going and saying, hey, I’m building APIs Does this mean I’m a marketing person? Well, not necessarily But you have to think about how people are going to be adopting these and every step that we’re going through And so he identified a set of metrics and a process that goes along with those metrics from finding developers, perhaps promoting your API It gets picked up on Hacker News, it gets found via Google searches, people are tweeting about it, what have you You’re getting them to go to a developer portal They’re signing up for it They’re using it within an application You’ve got active usage of that API within a number of apps that they shipped Perhaps you’ve got monetization associated with it This is going to be different for you Everybody’s API business is different Some of this won’t apply because you’re doing internal APIs But even in internal API ecosystems, there will be a set of metrics that you need to manage to This is what it means to be a service provider because you need to know your customers And in order to provide quality of service,

you need to be basically managing to these metrics So, hopefully these three forms of APIs make sense Hopefully for those of you who are involved in API projects, you’re able to think about and map what you’re doing to these types of APIs But what I want to talk about now is the ecosystem that the APIs sit within because this is where things really get interesting It’s not as simple as just going and saying my exposing my API is REST or I see this GRPC thing What are the technical considerations? Do I use a gateway with caching or not? All of those requirements are very important but it’s the ecosystems that your APIs are used at– used by that really is the whole reason why you’re doing this Most businesses do operate within ecosystems If you are a company that is becoming a digital business, you probably are already moving many of your supply chains, your value chains to be digitally mediated And that might be what’s driving your API thinking You may internally, within your IT processes, those represent a ecosystem and a value chain Most of what we’re talking about here is what the people who get all excited about this stuff– tends to be industry analysts, tends not so much to be developers and IT people– start talking about what your value chains are Well, hopefully this will serve as a primer to that You generally, when you’re building APIs, you’re going to be serving one or more ecosystems You’re going to have your internal ecosystems, those are sort of the APIs you share with your friends on your team, your internal developers, but this ranges out There might be partner ecosystems, there might be industry ecosystems, there might be public ecosystems These all have new considerations, new requirements Generally what we see is people sort of take this approach that I’m sorry to say is a little naive but that said, it’s the path everyone takes which is to go and say, well we’re going to build our APIs for internal usage and then we’re going to open them up to the outside world and it’ll be great Tends to– you tend to find, going back to my first slide, that a whole bunch of new considerations hit you over the head Hopefully halfway through this presentation, you’re starting to get an inkling of what some of those are But let’s talk about how these ecosystems are different If you’re doing ecosystems for your APIs as services, they’re predominantly internal These are what you’re doing to share and reuse the applications You build the systems, you build the back end resources, you build with other developers There’s a number of different approaches to this There’s the classic SOA approach Now, people who were SOA architects, SOA often– SOA stands for service oriented architecture Those of us who were around at the time go and say, well there’s nothing wrong with SOA It just simply says we should build as services that should be designed for reuse and you’ll get all sorts of productivity gains from that What we found was that most organizations ended up finding that you ended up having to impose a strong form of central governance in order to make this happen And there is a slippery slope that gets you there And I actually believe that about half the companies that are embracing micro services are going to end up there The way that you end up on the slippery slope is just going and saying, all we have to do is agree to a common set of conventions and all of our services can be reused Then nobody can agree with those common set of conventions are Hence the strong central governance model Now, what tends to work is this idea of an API management model An API model where you go and say look, I am going to expose the API in this way It will be a RESTful API so all of your clients can talk to it We are going to document it in standard ways We’re going to use the open API specification to document it And I’m going to stand up a developer portal and that’s where we’re going to find it And if that API doesn’t work the way you like, then what you should do is build your own client libraries to access it and perhaps use your own middleware, if you will Perhaps use Node.js if it’s something you want to build yourself to shape and convert that API to your use cases This tends to work very well We see that the interaction model– and that Node.js, as I said before, that Node.js model tends to be most closely associated with situations where these APIs need to be exposed for interaction purposes– this has a lot of interesting aspects to it

Mobile was the main driver for this but we now see things like conversational computing We see things like if you’re using Google Assistant, if you’re using Alexa, if you’re using any of these, Siri You’re seeing that you now have the ability, if you actually are delivering your services within standard interaction models, that they can be now used within these new types of conversational interaction models But we also see that there’s a lot of business drivers for this Retail people look at this and– folks in the retail industry they go and say, this is how I’m going to power omnichannel interaction models And you find that there’s this need to go and make your APIs available in standard ways so that you can actually talk to an e-commerce stack You might be able to talk to some form of direct response email interaction system and so on These tend to try– so supporting an ecosystem in this model tends to try or drive you to getting your APIs to fit certain patterns Right now, I say patterns, not standards There aren’t as many standards in this area as there could be And I think that both of these models, people tend not to think of themselves as participating in an ecosystem but I think increasingly you’re going to Let me talk about the product ecosystems because I think those are where things get the most interesting So within the product ecosystems, there’s a variety of different ways that people make their APIs available for use by other folks So basically, the idea here is that you’re taking your business capability, you’re packaging it up as a building block that other companies can go and embed in their business processes So, in terms of how you find these APIs, right now the predominant way that people find and interact with APIs is really sort of this one sided traditional commerce model where people go and discover the API, usually through a preexisting business relationship, or they are leveraging one of the major brands If I’m going and I need to send an SMS API, the API– there’s a set of choices, very well-known choices People learn about these through conventional marketing mechanisms But where things get more interesting is where you have this platform oriented approach where you actually can bring these APIs together and promote them and combine them in different ways Some of the examples of this that we see are for example, if you look at the partnership between MindBody which is– provides scheduling services for things like yoga studios, fitness centers, and so on, and what’s happened with Google Maps, this is an example of where you actually see a two sided relationship where both companies are actually sharing services and data between each other to create two different application experiences with a whole bunch of synergy between them So if I go to Google Maps and I go and I say I want to be able to go and book a session with a fitness instructor, I can go in and do that And Google Maps will call out to the MindBody API and they’ll be able to find the nearest fitness center where that’s available and directly book it That’s what we actually see in the right hand side with this screen that’s showing Reserve with Google I can get to this via the map user experience Now the flip side of this is that you’ve got Google Maps itself integrating with MindBody within its application So if MindBody is providing a private label application to a specific chain of fitness centers or to end users through the variety of channels that they do that, that users are able to go and get access to Google Maps with all of its location based services but in the context of this application experience So I think that what we’re seeing very successful within the API spaces where you’re able to basically cross-pollinate this experience by sharing data and process Some of the other forms of this slide are things that I think you’re going to see more of as we look out in the future of different forms of ecosystems One of the things I wanted to leave you with at the end of this presentation is this session that I just did is the first part of what we’re

calling The Connected Business Track here at Google Next to-morrow Chet Kapoor, CEO of Apigee at Google, is going to be talking about this vision of a connected business platform and how all of these different ecosystems power different types of business models And some of these business models that we see are actually these clearing houses and marketplaces for digital services and we’ll get into a lot more depth in that keynote tomorrow So, let me wrap this up So when we talk about API first and we talk about API centric thinking, hopefully this presentation has given you kind of this perspective of thinking about this outside in approach Even if your APIs are predominantly being used for internal purposes, it makes sense if your scope of reuse is wide enough If you are working with other development teams that are sort of beyond arm’s length where you’re not scrumming with them, where you’re not regularly interacting with them on Slack, you need to suddenly be thinking about your API consumers as customers Now if your APIs are designed for being APIs as products, then you definitely are going to have to think about this The competitiveness of your API, the success of your API as a product really will come down to the developer experience that you create and the quality of service that you provide Being able to identify what form of API that you’re building– is it simply designed for service reuse, is it a micro service, is it something that’s powering an interaction, or is it this API as a product– is going to help you think about how you organize your team around it Where it sits within your organization If it is designed for API as a service in that classical sense, maybe it is something that sits– if you’re in an IT organization– that sits within your classic integration center of excellence Although generally, we see and advocate people move away from that strong centralized model If it’s an interaction, it’s probably sitting with your front end team But even if it is, you now have considerations like how do I allow my front end team to go and build these interaction APIs and make sure they’re still secure? And if they’re APIs as a product, then do you have an API product manager? This last concept of managing your APIs as a digital partnership, super important This tends to be the light bulb moment for most folks when they think about their API programs, particularly when you’re bridging into trying to explain the value of APIs to the business folks in your organizations Suddenly when people understand an API is a digital partnership, they understand all the responsibilities and opportunities that go along with that And finally, regardless of how you are using APIs, you have to think about the ecosystems The nature of the ecosystems will be different for each one of those APIs But ultimately, the moment you’ve got one other person beyond the person who built their API– built that API using the API, you now have an ecosystem that you need to cultivate and grow and make successful