Write Production Quality Cloud Functions (Firebase Dev Summit 2017)

Write Production Quality Cloud Functions (Firebase Dev Summit 2017)


LAUREN LONG: Thanks for
being here, everybody. My name is Lauren. I’m here with my
coworker Thomas. We are both software
engineers that work on Cloud
Functions for Firebase. And today we will
be sharing with you how you can write
production-quality Cloud Functions code. To do that, we’ll dive into
three different topics. First, I’m going to
talk about why you even want to use Cloud Functions
in the first place. Next, I’ll invite Thomas up,
who will give us some tips, tricks, and also walk through
some tools for how you could be writing Cloud Functions code. Then I’ll be talking
about how you can test the code that you just
wrote using some tools we’ve released recently. So first, why Cloud
Functions in the first place? Today you may have
learned about some of the other Firebase
products that allow you to write an
app without a back end. So you learned about
Firestore for storing your data, Cloud
Storage for Firebase for storing pictures and videos. And these products allow you to
write your app without servers. But sometimes you actually do
need a server, for example, if you have secret
code that you don’t want visible to the client. This could be scoring logic
if you’re writing a game. You want to keep those
things on the server. Secondly, if you
are doing things that are very
computationally intensive, you want them on the
server so your clients can be performant and lightweight. And lastly, if you’re
writing multiple apps and multiple platforms, for
example, iOS, Android, and web, you don’t have to update all
of them to add a functionality or push a bug fix. By keeping code on the
server, you can also avoid lengthy app
review cycles, and you don’t have to wait for
users to update their apps. So you can host your own server. And in this kind of
a model, you have to define an interface
for communication between the server
and the client. Typically, the client will
be making HTTPS request to the server to
interact with it. You have to come up with
an interface, an API, for these two things
to work together. Cloud Functions for Firebase
is a managed back-end solution. And it offers a simpler
programming model. You don’t have to define
an API based on requests unless you want to. If you want to, you can
run it an event run manner. And what do I mean
by that exactly? So Cloud Function
is a snippet of code that runs on Google
servers in response to events happening in your
app, for example, a new user has just signed up, or
a node in the database has changed values. One of these events happen. That triggers your
function to run. Alternatively, you can still
trigger these functions through HTTPS requests. And this is a great way
to create web hooks. Web hooks are URLs that
third parties can hit. So for example, Slack,
Twilio, and Stripe are three of many,
many external services you can use web hooks with
to integrate into your app. So what are these events
that can trigger functions? Firestore, our
database solution, if something is written
into the Firestore, you can trigger
Function with that. As of today, you can
listen to crash analytics, crash events with Functions
and trigger code to run. You can run code when there’s
a new user that’s signed up. You can listen to
a database write in the real-time database. You can listen to an
analytics event being fired. And you file upload it in a
storage basket or, of course, an HTTPS request. One thing that you can
also do with HTTPS requests is you can connect
it with hosting. So Firebase Hosting’s
another product we offer. It does hosting for
your CSS and HTML files. Prior to Functions,
hosting could only host static websites–
websites that didn’t change depending on who the user is. But with Functions,
you can actually have your URLS that trigger
an HTTPS function to run, which then gives content
for hosting to render. This is great for rendering
dynamic visual content but also great for creating internal
APIs that your app can hit to get information. So why do you want to use
Cloud Functions to host your back-end code? The first reason is
it scales up and down. So if you’re getting a lot
of traffic to your app, you can trust that we will
spin up enough servers and serve all that
traffic for you. On the other hand, if you
have downtime in your app, you’re not paying for
what you’re not using. Secondly, because it
runs on Google servers, it’s already in a
trusted environment. So with no additional code,
you have authenticated access to Firebase and Google services. So you can do things like write
into your project’s database. You can access Google Cloud
APIs, such as our machine learning APIs. Lastly, it’s fully
managed by Google. Google operates
at a planet scale, and we do the DevOps for you. So you and your team do not
have to be burdened with that. So I just talked about what
are all the great reasons for using Cloud Functions. Next, Thomas will dive into
some concepts that will really help you write quality code. [APPLAUSE] THOMAS BOULDIN:
Thank you, Lauren. So Cloud Functions
lets you write code that runs in Node.js,
which means the tips I’m going to give you
today will work in the future for
other languages. But today we’re going
to take a very deep dive in node-specific
tips to remember when writing your Cloud Functions. Now, the Firebase design
process exposes us to a lot of customers, and
oftentimes those customers give us their code to look at
to help them debug or redesign. And we notice a pattern
as we work with customers. First, we noticed that
customers very often had an intuition for how
successful their code was going to be in production. Some devs would
tinker a little bit. They’d deploy to production. And they’d leave it
just still forever. Other devs could
iterate in their product and know with confidence that
when they deployed their code it would work. And they weren’t
just being foolhardy. We noticed when we
looked at these customers that they had high
confidence because they exhibited some very
common characteristics. We boil it down to three
traits of the most successful developers using
Cloud Functions. First of all, these
developers were able to express complex
thoughts much more clearly. When they asked us
to review their code, it was so much more
easy to read, especially their asynchronous programming. You see, bugs love to hide
in hard-to-read code, which means simple code
will have fewer bugs. Second, these developers tend
to lean more heavily on tools to develop their applications. And we’re going to use
three of those tools today. We’re going to use linters,
TypeScript, and IDs. And finally, I’m going
to hand it back to Lauren later to talk about
some of the tools we’ve used to test code
outside of production. If you notice, the
people who felt confident about their app’s
working production had actually tested it elsewhere
through unit testing, which we’ve already talked
about in previous talks, through testing in staging
apps, and through some of the tools that Lauren
will show you later. So many developers struggle
with asynchronous programming. We’ve boiled it down to what
we call the 1, 2, 3 test. How many of you here
might think if you’re glossing over this code
that it prints out 1, 2, 3? Well, it’s understandable
because these lines do come in order, 1, 2, 3. But this line right here
means that printing out two isn’t going to happen until
the rest of the code executes. Now, when I call out
the bug in isolation that might be a little easy. But how many of
you in 10 seconds can catch the incredibly
dangerous bug in this code? Did you catch it? There’s a error
return statement here. Whenever an error
happened in this code, it’s going to keep on running,
passing garbage data utilities, and calling the developer’s
call back more often. Bugs love to hide in what we
call the “pyramid of doom.” It’s when your code keeps on
nesting deeper and deeper as it progresses. This is a very,
very common place to have mistakes in your code,
which is why at Firebase we’ve been really pushing
developers to adopt promises in their code. But there’s just one problem. Your developers are so used
to having the pyramid of doom that you’ve been
reinventing it in promises. So here’s that same
code incorrectly written with promises. This is not how you
write promises, folks. This still has that
pyramid of doom, and it still has two
more dangerous bugs. Did you catch these ones? Well, promises
are return values. So you need to return them. That way the caller can
actually interact with them. This is how we know to keep
your Cloud Functions alive. If you don’t return the
promise, we’ll probably shut down your machine early. Now, second, there is a time and
a place for chained promises, or nested promises, but a
simple sequence of events is not one of them. This catch block
is attached only to task three, which means any
failure in task one and two is not going to be handled. So for everyone who’s
taking a picture, this is the correct way
you write that code– task one, then task two, then
task three, then all one line. No complex indentation. No pyramid of doom. This code is simpler. So it’s much more
likely to be correct. Now, there’s two
reasons that you should write your code like this. First, this is simpler. The amount of bugs
in your code is directly proportional to the
amount of code you’ve written– so write less code. Second, writing code
with idioms helps you write fewer bugs as well. Your brain recognizes
these patterns, which means you’ll notice when
the pattern is slightly off, and you probably have a bug. You’ll also be able to
quickly glance at the code and extract the meaning
out of this pattern. You start to recognize
idioms over and over again. But this can go overboard. Here’s code that I recognize. You shouldn’t read it. I don’t either. I just see this blob, and
I remember that everyone’s copied it from Stack Overflow. This is how you make a REST
request if you’re really trying to do it the hard way. So consider actually using
some tools to write your code. Here is that same code using a
native library request promise. This is much easier to read,
much more likely to be correct. So let’s continue
our talk about tools and go back to that sample. This function is correct. But it would be great
if we could do better, because almost half of
this code is boilerplate. Wouldn’t it be great if we
could focus on the things that mattered most? Well, today I want to give you
a sneak peek of the future. Node version 8 adds two new
keywords, await and async. Here’s how it works. First, you declare
a function as async. This means it’s going to return
a promise no matter what. It also lets you use
the await keyword. You stick await before any
command that returns a promise, and it tells node, hey, I need
a value here not a promise. Go do something
else for a while. And when that
promise is resolved, continue this code with the
value from that promise. So now we get to write
lines like this that looks so much more natural. Result equals await task 1,
not task 1 dot then result 1. So now our code
looks almost exactly like our synchronous code. You don’t have to keep
on asking yourself about that 1, 2, 3 test. There’s almost no
boilerplate at all. And await turns
rejected promises into real exceptions,
which means you can use try, catch, and finally like
you have for the last 10, 15, 20 years. There’s almost no boilerplate. And here’s another example. This code reads fairly simply. We’re looping on an
operation until it’s done and printing it when
we’ve finished waiting. If you wrote this
with raw promises, it would be much more complex. That while loop
would be recursive. Try finally is actually pretty
subtle with raw promises. We fixed a bug recently
in our own SDKs because we implemented
finally wrong. So why am I telling
you about things that aren’t available yet? Why are we in the tool
section of this talk? Because today I
want to talk to you a little bit about TypeScript. Now, those of you who
know me, know that I’m a huge fan of TypeScript. It’s a language
written by Microsoft that extends JavaScript
with type information, which helps you write better apps. First of all, because we
have type information, your IDE can give you
great code complete. And when you screw up, you
actually get compiler errors. No more deploying and finding
something is not a function. Have you done that? Also, because we have
type information, our linters are so much better. If you’ve ever forgotten
to return a promise, imagine having a compiler
automatically check that for you. And finally, TypeScript
gives you features of Node.js that aren’t available yet. So I’m going to do today’s
demo in TypeScript. And we’re going to see some
of these tools in practice with an app that I like
to call Alone Together. This demonstrates my
favorite mathematical paradox called the friendship paradox. It says that most
people are less popular than their friends. That doesn’t make
a lot of sense. But you can see an example here. We have a social graph
with three people. One person knows two. The average person
has 1.3 friends, and 2/3 are below average. So I like this example. Because for people who
grew up as loners, like me, it lets me know that I’m
not alone in being a loner. So to do this, we’re going
to use a Twitter API. We’re going to find out
how many people follow you, find out who you follow, and
how many people follow them and do some stats. So it’s a bit of
a complex code, so let’s go through the
architecture first. Well start with the user name
and fetch the user’s profile. Another request can give us
a list of IDs of the people we follow. Now, there might be hundreds
and thousands of them, and we want to turn that into
the friend’s follower account. But we can’t make thousands
of API requests at once. We’re going to get rate limited. So Twitter’s given
us a batch API, but that has a
size limit as well. So we need to take our
big array and break it into an array of batches. So to do that, before
we call GET user/lookup, we will call chunk. This will break up our
array into subarrays of a maximum size. From there, we can do
this look up in parallel and use promise.all to wait for
all the batches to complete. Now, the opposite
of chunk is flatMap. It will take our array
of arrays and give us one array of results. And now that we have our
profile and the follower account of our friends,
we can calculate some fun and existential statistics. So let’s go ahead and
see this in action. So when we write TypeScript,
one of the first things you’ll notice is that imports
are a little bit different. It’s because the
require statement is very node specific– before
JavaScript invented modules. Now JavaScript has and
TypeScript gives us access to the latest features. So we use things like import. This is the syntax. It’s equivalent to the require
statement you’re used to. But there are some more advanced
syntaxes you can look up later. Next, we will initialize
the M and SDK. The Firebase functions
runtime automatically gives you the information
you need to do that. Now, stylistically, I like
to write helper functions for any API call
that really give us the meat of what we care about
to make our code more readable. So I’m going to create
one function that looks up a profile– or looks up a list of
IDs, list profiles, and just extracts
the IDs out of it. Next the lookup of user. Since this was a batch
request, my helper just extracts the first
one from the batch. getFollowerCount will
look up a whole profile and just extract
the follower ideas. So now, with these,
the rest of my code will be a lot easier to read,
and each of these functions are easy to look
at independently. And finally, since I
believe in testing, I’m going to make my
batch size an export. This means when I’m
writing a unit test, I can change the value
of the batch size, and I don’t have to create mock
data of over 100 users just to test that my
batch chunking works. So let’s get started
with the HTTP function. Here’s how you write an
HTTP function in Firebase, function.https.onrequest. Now we had the ability
to handle a raw request response express object. And because we use
the async keyword, we can use await inside
this function, which means I can have my error
handling with just a try catch block. If any error
happens, log it so I can check out the production
logs later and do a 504. Now we’re going to
start with the user from the query parameter. And I’ve written a small helper
to authenticate the Twitter API. So now let’s go
through our workflow. First, I get my profile. Now I find the
IDs of my friends, break them into batches. And now I’m going to map
each batch into a promise to fetch the results for
that batch, so map and then to getFollowerCount. So now we have an array
of promises of results. Let’s await to get the results. Now we can combine
them with that flatMap. And we can calculate
our statistics. So now I have my
friend count, my fames, how many people follow me,
the probability distribution, the number of friends who
are more famous than I am, and the ratio. So if more than 0.5– if this ratio’s higher than
0.5, more than 50% of my friends are more famous than me. So let’s log that
just to make sure we have some interesting stats when
we look at our production logs. We’ll save it to the database. Oops. And since this is
an HTTP function, we need to return something. We’ll return a shorter
version of that. So now we hand out the
user name and how many or what percentage
of their friends are more famous than they are. So that’s our HTTP function. But you’ll notice I wrote this
information to the database. The reason why is it’s not fun
to just count how popular I am, but how well I fit in with
the rest of the world. So I’m going to write a
database function that listens to this
node here, and it will use that to update
some aggregate statistics. So this is what a database
function looks like, functions.database.ref. I write a pattern to
match all the documents that I want to match and then
the event type, on create. Whenever a new user
adds their information, we’ll update the
aggregate statistics. So as I said before, I am
more famous than my friends if fewer than half of my
friends are more famous than me. Makes sense? Now, because many
users are going to try to update the same
node, we’ll use a transaction. And then we’ll make sure
we initialize our variable in case this is the first time
that that transaction is run. We’ll either update the
more famous than peers count or the less
famous than peers count. Always update the total and
always calculate the new ratio. And just to make things
mildly more interesting, we will log what
the new stats are. So now we have completed
HTTP and database function. We’re going to hand
it back to Lauren to show you how you can
actually test this even outside of production. LAUREN LONG: Great,
thank you, Thomas. Awesome. So Thomas just walked us through
some tools and tips and tricks for how you can be
writing your functions. Next, I want to talk about
how you can test and debug these functions. So there are two main
parts of testing. The first is writing unit tests. And we don’t have enough
time to dive into it today, but at the end, I’ll
give you some resources you can follow up on later. The second part of testing
is tinkering with your code while you’re still
developing it. And this is what Thomas
talked about earlier when he talked about playing with
your code before production. So one tool that
Firebase offers for this is a local server to emulate
your HTTPS functions. So this looks at
your project, sees if there are any functions that
are listening to a request. And if there are, serve
them up on local URLs. And you can go to a web
browser to hit these URLs, use a cURL command. Or if you’re doing an
integration between hosting and function, where
hosting URLs are getting redirected
to HTTPS functions, you can serve them
both at the same time. And any interactions you
have with the hosted site will also trigger
your local functions. The cool thing about this
is your local function will actually be talking
to your production app and make real requests
to external APIs, like the Twitter API
we’re using today. How you use this is through
the Firebase Command Line Interface, the Firebase CLI. You run firebase serve dash dash
only functions if you’re only serving up functions. Or you add comma hosting if
you’re using both of them at the same time. So let’s see this in action. And Thomas will actually
help us out here. So we already ran the firebase
serve dash dash only hosting functions command on the
terminal on the left. We see that we have a
local website on the right, as well as a function at
a URL that’s printed here. So this is our
aggregate statistics. Before this talk, we put in some
Twitter handles of Firebasers. So you can see that there
are 12 people that are less famous than their friends. So these 12 people
have less followers than half of the people
that they follow. And there were four
people that were actually more famous than their friends. So these are more popular
Firebasers among us. They have more followers
than half of the people that they follow. And you can see that
25% of the people are more famous
than their friends. So we build this
app by integrating hosting with functions. So Thomas right now is going
to click on the Add Your Stat button. It’s going to ask him to log
in to his Twitter account so we can pull up
his statistics. Now, when this login
is done, the app is making a request to an
internal URL, this slash rating URL, which triggers our
local function to run. And we can see logs
here on the left. So everything after info
is from the function. So we can see that the
function was triggered. It started execution. We used authentication. That was the Twitter login. We printed the
full stats, so that was the console log in our code,
the console logs show up here. We see his Twitter
handle is in-lined. He has 869 followers, and
he follows 438 people. And there are some
more statistics, but the most interesting
one is that 62% of the people that he follows
have more followers than him. So Thomas is contributing
to our less famous stat. You see on the right that
went from 12 up to 13. And at the line at the
very bottom here in orange shows him that
62% of his friends are more famous than him. Let’s switch back to the slides. So we were able to test
out our HTTPS function. But if you remember, there
were actually more functions in our directory. There was a database function. So how do we test that? We have another tool called
the functions shell, which will emulate all of the
functions in your project, no matter what trigger
type they listen to. This is a full node
shell that is interactive that you can use regular
JavaScript code with. How you use it is, again,
through the Firebase CLI by running Firebase
experimental functions shell. So let’s see this in action. So we already ran
the command here, and it emulated two functions,
the aggregate function as well as the rating function,
which we already tested. So we won’t test that again. And if you remember
back to the code, the aggregate function
is a database function that’s looking at any
new database writes to stats for a user
that just used the app. So Thomas just used it. So let’s actually
for test data use the data that was just printed
out from the previous function. So I’m going to copy this. So remember, this is
an actual node shell, so we can declare
variables here. So we’ve got data right here. And in the shell, each of the
functions that we emulated, the names of them actually
becomes a function we can call. And calling that function
invokes a local function. And the parameter we would
put in is the test data. So we see that the function
was successfully invoked, and the console logs
from our functions are showing up here as well. So we are including his stats. We can double check here
that, yep, our function pulled the right stat. Is he more famous
than his friends? False, unfortunately. And because of that, we
are updating our stats from 13 to 14 for the number
of Firebasers less famous than their peers. And that brings us
to the ratio of 22%. Let’s jump back to the slides. So I’m sure you’re all
very anxious to see how unpopular you are. So you can pull out your
phone, your computer. If you’re on live stream,
you can do this as well. Go to
alone-together.firebaseapp.com. It’s going to ask you to
log in with your Twitter. So do that, and it’ll pull
your follower account, look at the people
that you follow, look at their follower count,
and calculate statistics. Can we switch back to the
demo computer, please? Wow, nobody’s more famous yet. Come on. So I see that as
an audience, there are 11% of us that are more
famous than our friends. We can switch back
to the slides. So you’ve just learned why
call functions for Firebase is an excellent choice
for writing server code. We talked about some tips for
writing production quality functions, such as
avoiding nested promises, using await, async,
and using TypeScript. We also learned
about how you can be testing your
functions locally through Firebase Serve
and the functions shell. If you’re interested in the
source code of the app we just played with, you
can find it at alone together.firebaseapp.com/source. So I’ll give you a second
to take a photo of that. And if you want to
learn even more, these are some resources
to help you out. So I mentioned that
testing is something that also involves
unit testing, which we didn’t dive into today. There is a talk that Thomas
actually did at Google I/O earlier this year
that you can find on YouTube titled “Cloud
Functions, Testability, and Open Source.” You can go on our Documentation
page, firebase.google. com/docs/functions. And both of us will be at
the Ask Firebase lounge right after this talk to
answer any additional questions you have. Thank you for listening. Here are our Twitter
handles if you want to help us beat
the friendship paradox. Thank you. [APPLAUSE] [MUSIC PLAYING]

20 thoughts on “Write Production Quality Cloud Functions (Firebase Dev Summit 2017)

  1. hello I love the ease of firebase functions, but I would like to be able to program anywhere I am and I have not found how to do it. They should have how to do this from an APP, or from the web. thus nothing would program the code, I give it to save and it should be published. Thank you

  2. I'm not a fan of typescript. Tried using it with Angular 2-4 and became frustrated and went back to vanilla JS. Much happier and get more done.

  3. #AskFirebase Hi! I tried to use async functions but does not compile base on that firebase cloud has Node version 6.x.. do you know when will be available Node version 8 in Google Functions?

  4. I know I shouldn't ask this question here but I have been trying to get an answer from Google Cloud staff for months (literally) so this is my desperate attempt here:
    What is the ETA for Cloud Functions to be available in EU regions?

  5. The code at 13:05 mutates state and could mask bugs in more complex situations. Perhaps the observable pattern is a healthier approach (i.e., expand using RxJs). https://stackoverflow.com/questions/44097231/rxjs-while-loop-for-paging

  6. i need to send daily mail with particular time to all my users is it possible to do in firebase cloud function #Firebase @Laurenlong

  7. FYI, 'Alone Together' is a 1970 album by Dave Mason. It's best known single is 'Only You Know and I Know'.

  8. Yay Typescript, so all the code examples you find will not actually work. That was one of the great advantages of Coffeescript if I remember ancient history correctly. But this time it was invented by MS so surely it will not be a technological dead end. 😉

  9. const complex = () => task1()
    .then(task2)
    .then(task3)
    .then(errorHandler)

    Is technically shorter than

    async function complex() {
    try {
    const result1 = await task1();
    const result2 = await task2(result1);
    return task3(result2);
    } catch (err) {
    return errorHandler(err);
    }
    }

    All in all, `try/catch` of async is really dropping the ball.
    Can't wait for JavaScript to adopt a Result datatype.

  10. If you organize your functions in folders like: users/addUser.js, users/renameUser.js, orders/addOrder.js, orders/search.js,
    can the URLs be made to include the folder name like: http://mybaseurl/users/addUser, http://mybaseurl/orders/search ?

Leave a Reply

Your email address will not be published. Required fields are marked *