Integrate PayPal Webhook Notifications with Rails

As part of a recent project we needed to receive and process webhook notifications issued by PayPal. In this article I outline some techniques and code snippets which might prove useful to someone picking up a similar task.

What is a webhook?

Webhooks are a standard method for sending data between different applications and platforms. One platform will be responsible for issuing the notification and the other platform, or app, must be ready to receive the notification.

In our case PayPal is the platform responsible for issuing the notification. It can generate a bunch of webhook notifications for any events that you might be interested in; for example, when someone makes a purchase on your site, when a payment is refused, when a subscription renewal fails etc. One can appreciate how useful this can be, allowing us to execute arbitrary business logic on the back of such events.

But how is this webhook magic achieved? Generally you will need to setup your webhook with the platform (in our case PayPal), which basically means telling PayPal the URL to which they should send this data, once the event occurs. Once PayPal knows the URL to send to we will need to make sure that we have a server endpoint in place, ready to receive the notification event.

Getting started with webhooks

To get started with processing these notifications we first need to tell PayPal where to send the data. We do this by configuring our webhook on the PayPal developer dashboard. You should navigate to the Apps & Credentials section and select the application name of the PayPal application for which you wish to configure the webhook, in our case we are choosing the VectorLogic Demo app.

Accessing the application configuration screen on the PayPal Developer console.

We navigate to this VectorLogic Demo configuration screen and scroll to the bottom to reveal the Webhooks section, with the option to Add Webhook. When we add a new webhook we only need to define two things:

which events we want to be notified about, and
the URL to which the data should be sent, when the event occurs.

For the sake of demonstration we will consider a generic notification endpoint at /pay_pal/notifications, and this endpoint will process notifications raised for all events. However, you can see how it is possible to set up multiple webhooks via the PayPal dashboard, where different URLs are exposed to handle a different subsets of events. Indeed, these endpoints could potentially exist on different servers and even different domains. But in our case we'll keep it simple: one URL handling all events.

Adding a new webhook to the PayPal application.

And this is all that is required to get our webhook setup on PayPal. The dashboard will assign a unique ID for the webhook that we have set up. Take a note of this as we will use this :webhook_id later in the process.

The PayPal infrastructure will now take care of POSTing data to our server endpoint when any of the specified events occur. Now we need to actually expose this URL on our server. First we add the new route to our Rails project. In config/routes.rb we add the following:

    
Rails.application.routes.draw do
  …
  namespace :pay_pal do
    resources :notifications, only: [:create]
  end
end

And we create a new PayPal::NotificationsController to implement the #create action that we expose:

    
class PayPal::NotificationsController < ApplicationController
  skip_before_action :verify_authenticity_token, only: [:create]

  def create
    head :ok
  end
end

And this is all we really need for our server endpoint, from PayPal's perspective. As PayPal will not be including an CSRF token on the request we should skip the CSRF checks on create action. PayPal will POST the event data to our /pay_pal/notifcations URL, which we have now exposed via routes.rb. Our controller action just accepts the POSTed data, does nothing, and returns a 200 response. Once PayPal receives the 200 response it will consider the notification received and will not need to resend the data.

This is an important point and worth an additional remark. As application developers we should be aware that the internet is an unreliable place, sometimes connectivity drops, sometimes servers are offline, in general we need to design for failure. The PayPal webhook system is no different. PayPal will attempt to send the webhook notification; if it receives a 200 response it will know that the notification has been successfully processed and does not need to be resent. If it does not receive a 200 response PayPal will reattempt to resend the notification multiple times, until a successful response is received.

From the docs, [1]:

If your app responds with any other status code, PayPal tries to resend the notification message 25 times over the course of three days.

Presumably the rate of these resends is determined by some form of exponential back-off. In any case, this means that if we fail to process the event at the first attempt we will get another try. This is reassuring, however there is a flip side. Suppose we send our 200 response but PayPal doesn't receive it? Or what if our processing of the event times-out before the response is received by PayPal. In such cases PayPal will send the same event again. We need to be prepared to receive the same notification more than once. Our webhook endpoint should be prepared for such cases and should handle them sensibly.

With that out of the way, let's dig a bit deeper into the processing of our notifications.

Processing webhook events

Our dirt-simple PayPal::NotifcationsController is good from the PayPal perspective (a speedy 200 response and everyone is happy). However, in the real world we want to actually process the notification in some way, to benefit our application. This could mean updating the user record in our database, emailing the user impacted, storing the notification, launching a satellite, whatever.

Typically we will want our webhook endpoint to parse the data received from PayPal and update our system in some way, based on the type of event and the data received. The structure of the data received from PayPal, on each event, can be inferred from the API documentation, or you can actually trigger simulated events to your endpoint using the webhooks simulator. There can be quite a bit included in the notification payload, but for our purposes we just remark that the POSTed data will include an :id key, which represents the unique ID which PayPal uses to identify the notification. We will use this ID in our controller endpoint.

Let's now consider a more useful #create action; I will provide the code listing first and then break it down in detail after:

    
class PayPal::NotificationsController < ApplicationController
  skip_before_action :verify_authenticity_token, only: [:create]

  SUPPORTED_EVENTS = %w(
    PAYMENT.SALE.COMPLETED
    BILLING.SUBSCRIPTION.ACTIVATED
    BILLING.SUBSCRIPTION.CANCELLED
    BILLING.SUBSCRIPTION.CREATED
    BILLING.SUBSCRIPTION.EXPIRED
    BILLING.SUBSCRIPTION.PAYMENT.FAILED
    BILLING.SUBSCRIPTION.SUSPENDED
    BILLING.SUBSCRIPTION.UPDATED
  )

  before_action :ensure_supported_event

  def create
    unless get_notification
      verify_notification!
      ActiveRecord::Base.transaction do 
        create_notification.process!
      end
    end
    head :ok
  rescue PayPal::WebhookError => e
    Rails.logger.error("Something went wrong")
    head :unprocessable_entity
  end

  private

  def create_notification
    PayPal::Notification.create!({
      …
      pay_pal_notification_id: params.fetch("id"),
      …
    })
  end

  def ensure_supported_event
    return if SUPPORTED_EVENTS.include?(params.fetch("event_type"))
    head :ok
  end

  def verify_notification!
    # Implementation to follow
  end

  def get_notification
    @_notification ||= Payments::PayPal::Notification.where(pay_pal_notification_id: params.fetch("id")).first
  end
end

Lets pull out a couple of salient aspects of this implementation:

Our webhook has been configured to accept events of all types, but let us now presume that we only want to handle a subset of the PayPal events in this controller. As a sanity check we maintain a hardcoded list of event types, SUPPORTED_EVENTS, that we wish to process. A before_action is used to to check that the event_type on the incoming payload matches the supported events, if we receive an event we do not intend to process we will automatically return a 200 response:
```
        
def ensure_supported_event
  return if SUPPORTED_EVENTS.include?(params.fetch("event_type"))
  head :ok
end
        
      
```
Within the #create action the first step is to call get_notification. If this call retrieves the notification from our DB we know that this notification has already been processed and stored, so we automatically return our 200 response.
Next we call verifiy_notification!, which is responsible for ensuring that this notification has really been sent by PayPal and hasn't been tampered with. We will examine this step in more detail shortly.
We then create the PayPal::Notification on our DB and call process! on the instance created. We will gloss over this part as the details will depend upon your specific application and what actions need to be completed in response to each PayPal event. However, we note that each notification will lead to a new instance stored on the DB; an error in creating the PayPal::Notification or in running the process! method should lead to a custom PayPal::WebhookError being raised. This will cause the enclosing transaction to be rolled-back, and our PayPal::Notification will not be persisted. By virtue of this transaction block, and the get_notification check at the start of the action, we should ensure that we only process each event once, even if we receive it multiple times from PayPal.
Presuming there are no issues we will return a 200 response to PayPal, communicating that the notification has been successfully processed. Otherwise we will return a 422 response.

Verifying a notification

Looking at our PayPal::NotificationsController you may have noticed that we make no attempt to authenticate the request, after all this request is not coming from a logged-in user. However, this means that anyone in the whole-wide-internet could POST data to this endpoint. Indeed, if someone wanted to be nasty they could maliciously mock the data payload we expect from PayPal in the hope that we will unwittingly process their request and update the payments or subscriptions on our system … or launch a satellite. We need this endpoint to be public so that PayPal can communicate with us, but how do we protect ourselves against such exploits?

For most webhook-based systems we achieve this by verifying the notification we receive. Some details for how we achieve this are provided by the PayPal docs. This provides some technical details on the verification process, and a Java implementation, but this does not translate easily for our Ruby application.

The first place we might look is for a gem offering PayPal integration. According to this reference the supported option would be the PayPal-Ruby-SDK. However, if we take a look at the GitHub repo it looks like this project has been archived, and has not received updates in over 2 years. Concerned about adding a dependency which is no longer being maintained, I preferred to try and implement this verification by-hand. This turned out to be a little more complicated than I had anticipated, and requiring hands-on usage of a number of different OpenSSL classes and methods. However, using the details in the PayPal docs and with reference to existing PayPal-Ruby-SDK gem, I had some pretty comprehensive guide rails. Let's look at how we can verify these PayPal notifications by-hand.

Manual verification

We will create a class to carry out the verification process, PayPal::NotificationVerifier. We will design the interface to this class so that it can be easily plugged into the verify_notification! step in our controller, as follows:

    
  def verify_notification!
    PayPal::NotificationVerifier.new({
      cert_url: request.headers.fetch("HTTP_PAYPAL_CERT_URL"),
      transmission_time: request.headers.fetch("HTTP_PAYPAL_TRANSMISSION_TIME"),
      transmission_id: request.headers.fetch("HTTP_PAYPAL_TRANSMISSION_ID"),
      transmission_sig: request.headers.fetch("HTTP_PAYPAL_TRANSMISSION_SIG"),
      payload: request.body.read,
      auth_algo: request.headers.fetch("HTTP_PAYPAL_AUTH_ALGO")
    }).verify!
  end

We pull a number of values from the request headers, as specified in the PayPal docs. One of these values, PAYPAL-TRANSMISSION-SIG, is the actual signature calculated by PayPal before sending the notification. Our goal is to use asymmentric public key encryption to calculate this signature on our side, and verify that it matches. If our generated signature matches we can be sure that the message originated from PayPal and has not been tampered with. The values that we pass into our verifier class are:

PAYPAL-TRANSMISSION-SIG: The signature as generated by PayPal. We calculate the signature ourselves and verify against this value to ensure they match.
PAYPAL-CERT-URL: The public key certificate that we should use in the verification
PAYPAL-TRANSMISSION-TIME: Timestamp generated by PayPal representing when the notiifcation is sent out
PAYPAL-TRANSMISSION-ID: A unique ID for this notification
The body of the notification request
PAYPAL-AUTH-ALGO: The algorithm used to sign the request.

The class implementation will look like this:

    
class PayPal::NotificationVerifier 
  attr_accessor :transmission_id,
    :transmission_signature,
    :transmission_time,
    :cert_uri,
    :auth_algo,
    :payload

  def initialize(transmission_id: nil,
                 transmission_sig: nil,
                 transmission_time: nil,
                 cert_url: nil,
                 payload: nil,
                 auth_algo: "SHA256")
    raise ArgumentError, "Must supply :transmission_id" unless transmission_id
    raise ArgumentError, "Must supply :transmission_sig" unless transmission_sig
    raise ArgumentError, "Must supply :transmission_time" unless transmission_time
    raise ArgumentError, "Must supply :cert_url" unless cert_url
    raise ArgumentError, "Must supply :payload" unless payload

    @cert_uri = URI.parse(cert_url)
    raise Payments::PayPal::WebhookError, "Invalid :cert_url" unless @cert_uri.is_a?(URI::HTTPS)
    @transmission_time = transmission_time
    @transmission_id = transmission_id
    @transmission_signature = transmission_sig
    @auth_algo = map_algo(auth_algo)
    @payload = payload
  end

  def verify!
    raise PayPal::WebhookError, "Unable to verify message" unless is_valid?
  end

  private

  def map_algo(algo)
    case algo
    when /^SHA256/
      "sha256"
    else
      raise PayPal::WebhookError, "Unsupported digest algorithm: #{algo}"
    end
  end

  def is_valid?
    get_cert.public_key.verify(get_digest, signature_base64, signature_input)
  end

  def get_cert
    return @_cert if @_cert
    if !cert_uri.host.match?(/\.paypal\.com$/) 
      raise PayPal::WebhookError, ":cert_url is not on paypal.com"
    end
    data = Net::HTTP.get_response(cert_uri)
    @_cert = OpenSSL::X509::Certificate.new data.body
  end

  def get_digest
    OpenSSL::Digest.new(auth_algo).update(signature_input)
  end

  def signature_base64
    Base64.decode64(transmission_signature).force_encoding('UTF-8')
  end

  def signature_input
    [ transmission_id,
      transmission_time,
      Rails.configuration.pay_pal[:webhook_id],
      payload_crc32 ].join("|")
  end

  def payload_crc32
    Zlib::crc32(payload.force_encoding("UTF-8")).to_s
  end
end

Our PayPal::NotificationVerifier initializer will check the presence of all the required parameters and raise an ArgumentError should any of these be absent. The one optional parameter is the auth_algo, which we default to SHA256. In principle, our PayPal notification can be signed using other algorithms, but in practice I have only come across the PAYPAL-AUTH-ALGO header with a value of SHA256withRSA. What's more, I am not sure how other potential values would map to OpenSSL algorithms. So the map_algo method will check that the auth_algo value passed begins with SHA256, in which case we will employ the OpenSSL SHA-256 algorithm to generate our signature digest. If a value for auth_algo is passed which does not match SHA256 we will raise an error.

During initialization we also check that the cert_url can be parsed by URI.parse. If this doesn't return a HTTPS URI we will, again, raise an error.

The public interface for the class is composed of a single method, verify!. This method will raise an error unless we determine that the signature passed is valid, and the main work is coordinated in the is_valid? method:

    
  def is_valid?
    get_cert.public_key.verify(get_digest, signature_base64, signature_input)
  end

get_cert will retrieve the public key from the cert_uri and use that to create an instance of OpenSSL::X509::Certificate. Before downloading the public key we do a quick sanity check to ensure that cert_uri is hosted on paypal.com, or a subdomain thereof.

We can use the public_key method on our certificate to retrieve the associated public key, in the form of a OpenSSL::PKey::RSA. Instances of PKey expose a verify method which we use to check the signature, that we calculate, against the value passed in on the PayPal notification.

As outlined in the PayPal docs, the input to signature algorithm is constructed from four pipe-delimited components:

    
  def signature_input
    [ transmission_id,
      transmission_time,
      Rails.configuration.pay_pal[:webhook_id],
      payload_crc32 ].join("|")
  end

Here the transmission_id and transmission_time are just the exact values passed in the request headers, and the :webhook_id we recorded previously from the PayPal dashboard - I have opted to store this value in application config for convenience. The final value is payload_crc32, which is a CRC32 digest of the body of the notification request we received. In Ruby we can calculate this digest as follows:

    
Zlib::crc32(payload.force_encoding("UTF-8")).to_s

This signature_input can be considered as the document that has been signed. We will create a digest of this document using the algorithm indicated in the PayPal request header (i.e. SHA-256). The OpenSSL::Digest class provides us the necessary tools for the job. With this digest calculated we can finally run the verify check on the OpenSSL::PKey::RSA; this requires that we pass the message digest, the signature value transmitted by PayPal in the request (but in base-64 encoded form) along with the signature_input before calculating the digest. This method call will return a true or false value, and will untimately dictate if our PayPal::NotificationVerifier#verify! passes, or if it raises an error.

And that's it! It wasn't quick, but we finally got there.

Verify via API

So after doing this by-hand I got to thinking that this was quite an involved process for what must be a common PayPal integration requirement. Things would have been easier if we had just incorporated the PayPal-Ruby-SDK gem, but who wants to use an unmaintained dependency? However, after a second look over the PayPal docs, it seemed that there was another way.

The webhook API docs reveal that PayPal offers an endpoint where you can POST the details of the notification you have received and the API will verify the notification for you. Simple as that.

This is probably the preferred method, as it doesn't require that we get down-and-dirty with the OpenSSL classes. This also plugs that auth_algo hole, where we are basically assuming that the algorithm will always be SHA256withRSA. However, it does present its own drawbacks. For example, we will need to do some additional work to request an OAuth token before calling the verify-webhook-signature API endpoint, and each of these calls does represent an additional network roundtrip. But all this being said, had I encountered this API endpoint first I may well have opted to verify the PayPal notifications in this manner.

Summary

In this post we discussed some of the technicalities with integrating a Rails application with PayPal webhook notifications. We provided code samples demonstarting how this could be achieved and we walked through the details of manually verifying the PayPal notification, to ensure the integrity of the messages we are processing.

I hope you found this useful. If you have any feedback or comments please let me know in the comments section.

References

Getting started with PayPal webhooks
AWS for Aerospace
PayPal docs for webhook events
A webhook simulator offered by PayPal
PayPal Ruby SDK
Using PayPal API to verify a notification
PKey#verify
OpenSSL::X509::Certificate ruby implementation
OpenSSL::PKey ruby library
OpenSSL::PKey::RSA ruby class
Wiki entry for CRC
Zlib::crc32 method in ruby