Use Case

Earned commission

Keep track precisely on how much you earned on a given timeframe

marketplaces
fintech

As a marketplace, you want to know precisely how much commission you're earning, either over a timeframe or at a specific point in time. As you grow and experiment with different commission models and the complexity grows, using a ledger as your single source of truth helps rationalize and build unified, real-time views of your earnings. You can connect Formance Ledger to Elasticsearch and Kibana to start getting insights into your commissions quickly.

Using this pattern

Choosing a Ledger Account Architecture

First, decide on a convention for your account architecture. Most commonly, you'll create a dedicated commission account for accruing commissions separately from other revenue streams. For example, here is a transaction in Numscript that records paying out a sale on order ID 1234 to seller ID 6789, along with a 15% commission on the sale:

send [USD/2 100] (
  source = @orders:1234
  destination = {
    15% to @platform:commission
    remaining to @users:6789
  }
)

By using naming conventions like this, we make it easy for Elasticsearch to index transactions based on what they represent, and not just to whom they are going. Then we can use Kibana to build a dashboard that gives real-time insight into your commissions by showing all money movements in and out of the dedicated commissions account.

Connecting Formance Ledger to Elasticsearch

Formance ships with native Kafka and HTTP publishing integrations, allowing you to sync Formance events with external systems. We'll use a handy tool called Benthos to create a data chain that moves our ledger transactions to Elasticsearch. Benthos facilitates connecting to Elasticsearch by receiving Ledger events over HTTP, and publishing those directly to Elasticsearch:

Ledger Webhook > Benthos > Elasticsearch 

Prerequisites

  • An Elasticsearch and Kibana deployment

  • An up-to-date Benthos binary

Setup

We'll first need to start our Formance Ledger with HTTP publishing enabled. You can enable this with the publisher-http-enabled and publisher-topic-mapping options, here passed as flag:

numary server start \
  --publisher-http-enabled \
  --publisher-topic-mapping='COMMITTED_TRANSACTIONS:http://localhost:3042/post'

Note that we'll post on localhost:3042, which is the endpoint we'll configure Benthos to bind to with the following config:

input:
  label: "ledger_hook"
  http_server:
    address: "localhost:3042"
    path: /post
    allowed_verbs:
      - POST
    timeout: 5s
 
pipeline:
  processors:
    - bloblang: |
        this.payload.transactions.map_each(tx -> tx.postings.map_each(p -> tx.without("postings").merge(p).merge({
            "ledger": this.ledger
        })))
    - unarchive:
        format: json_array
    - unarchive:
        format: json_array
 
output:
  label: ""
  elasticsearch:
    urls: ['http://localhost:9200']
    index: "demo-ledger-dashboards-1"
    id: ${!count("elastic_ids")}-${!timestamp_unix()}
    max_in_flight: 64

The configuration above creates a simple pipeline that deconstructs the ledger payload and creates an object to be indexed in Elastic for each transaction posting — the rationale for this mapping is simply to create something that will be easier to query and visualize with Kibana. You can save this config as config.yaml and start a Benthos process with it:

benthos -c config.yaml

We're all set! Let's commit a basic ledger transaction to verify. This transaction creates a new ledger called demo-ledger-dashboards-1, and moves some money from the outside world into a central bank account, and then from the central bank to a user's account:

curl -X POST \
  -H 'Content-Type: application/json' \
  -d '{
    "postings": [
      {
        "source": "world",
        "destination": "central_bank",
        "asset": "GEM",
        "amount": 100
      },
      {
        "source": "central_bank",
        "destination": "users:001",
        "asset": "GEM",
        "amount": 100
      }
    ],
    "metadata": {
        "description": "Wohoo! a first transaction."
    }
  }' http://localhost:3068/demo-ledger-dashboards-1/transactions

You can now head to the Kibana index management page, and should now see the demo-ledger-dashboards-1 index available for use.

Monitoring Commission Cash Flow in Kibana

Once our transactions data is in Elastic, we have everything we need at hand to use Kibana to create powerful real-time visualization of cash flow to the dedicated commissions account.

To start, create a new index pattern. Call it demo-ledger-dashboards-1, and use the field labeled timestamp for the timestamps (what else?).

image

Find your new index in the dashboard. By default, you'll see all the transactions; the histogram shows the volume of transactions over time. This isn't what we want, but we can see everything is hooked up.

image

Now, let's create the visualization. In the main menu bar, under "Analytics", choose "Visualize Library" to start creating a new visualization.

image

We're going to use the basic "Lens" visualization, so pick that.

image

Now, we're at the visualization editor. The key field for our purposes is amount, which is the amount each transaction moves in our ledger. Drag amount into the middle of the visualization builder.

image

So far so good, but this visualization is showing us the cash flowing through the entire ledger, not just our commissions account. And it's showing us by default the median amount of the transfers each time period. Let's chance that to sum, so we can see totals instead of measures of central tendency. Click on the label under "Vertical axis" on the right-hand panel, and choose "Sum".

image

Finally, let's make sure we're only looking at commissions being paid. At the top left, click "Add Filter". Select destination.keyword as the field, is as the operator, and platform:commission as the value.

image

And now, finally, we have what we wanted: A dashboard showing the cashflow into the commissions account, second-by-second. We can save this visualization so we can view and share with the rest of the team!

image
Use Cases

Use multi-destination to collect fees on your ledger transactions

Split a customer payment to multiple parties with templates

Model funds in flight on your ledger to enable deferred payouts and fund pending processing

900+ payments innovators building on Formance and sharing best practices.