Requirements

  • 3 million users
  • 5000 stocks + 250 global stocks
  • a user gets notified about the price change when
    1. subscribing the stock
    2. the stock has 5% or 10% changes
    3. since a) the last week or b) the last day
  • extensibility. may support other kinds of notifications like breaking news, earnings call, etc.

Sketching out the Architecture

Contexts:

  • What is clearing? Clearing is the procedure by which financial trades settle – that is, the correct and timely transfer of funds to the seller and securities to the buyer. Often with clearing, a specialized organization acts as an intermediary known as a clearinghouse.
  • What is a stock exchange? A facility where stock brokers and traders can buy and sell securities.

Apple Push Notification service
(APNs)
Apple Push Notification service<br>(APNs)
Google Firebase Cloud Messaging
(FCM)
Google Firebase Cloud Messaging<br>(FCM)
Email Services
AWS SES /sendgrid/etc
Email Services<br>AWS SES /sendgrid/etc
notifier
notifier
External Vendors

Market Prices
[Not supported by viewer]
Robinhood App
Robinhood App
API Gateway
API Gateway
Reverse Proxy
Reverse Proxy
batch write
batch write
price
ticker
[Not supported by viewer]
Time-series DB
influx or prometheus
Time-series DB<br>influx or prometheus
Tick every 5 mins
[Not supported by viewer]
periorical read
periorical read
price
watcher
price<br>watcher
User Settings
User Settings
Notification Queue
Notification Queue
throttler cache
throttler cache
cronjob
cronjob

What are those components and how do they interact with each other?

  • Price ticker
    • data fetching policies
      • option 1 preliminary: fetches data every 5 mins and flush into the time-series database in batches.
      • option 2 advanced: nowadays external systems usually push data directly so that we do not have to pull all the time.
    • ~6000 points per request or per price change.
    • data retention of 1 week, because this is just the speeding layer of the lambda architecture.
  • Price watcher
    • read the data ranging from last week or last 24 hours for each stock.
    • calculate if the fluctuation exceeds 5% or 10% in those two time spans. we get tuples like (stock, up 5%, 1 week).
      • corner case: should we normalize the price data? for example, some abnormal price like someone sold UBER mistakenly for $1 USD.
    • ratelimit (because 5% or 10% delta may occur many times within one day), and then emit an event PRICE_CHANGE(STOCK_CODE, timeSpan, percentage) to the notification queue.
  • Periodical triggers are cron jobs, e.g. Airflow, Cadence.
  • notification queue
    • may not necessarily be introduced in the first place when users and stocks are small.
    • may accept generic messaging event, like PRICE_CHANGE, EARNINGS_CALL, BREAKING_NEWS, etc.
  • Notifier
    • subscribe the notification queue to get the event
    • and then fetch who to notify from the user settings service
    • finally based on user settings, send out messages through APNs, FCM or AWS SES.

Designing Stock Exchange

2149 2019-08-12 09:50

Requirements

  • order-matching system for buy and sell orders. Types of orders:
    • Market Orders
    • Limit Orders
    • Stop-Loss Orders
    • Fill-or-Kill Orders
    • Duration of Orders
  • high availability and low latency for millions of users
    • async design - use messaging queue extensively (btw. side-effect: engineers work on one service pub to a queue and does not even know where exactly is the downstream service and hence cannot do evil.)

Architecture

Reverse Proxy
Reverse Proxy
API Gateway
API Gateway
Order Matching
Order Matching
User Store
User Store
settle
settle
Orders
Orders
Stock Meta
Stock Meta
auth
auth
Cache
Cache
Balances & Bookkeeping
Balances & Bookkeeping
external pricing
external pricing
clearing
house
clearing<br>house
Bank, ACH, Visa, etc
Bank, ACH, Visa, etc
Payment
Payment
Audit & Report
Audit & Report

Components and How do they interact with each other.

order matching system

  • shard by stock code
  • order’s basic data model (other metadata are omitted): Order(id, stock, side, time, qty, price)
  • the core abstraction of the order book is the matching algorithm. there are a bunch of matching algorithms(ref to stackoverflow, ref to medium)
  • example 1: price-time FIFO - a kind of 2D vector cast or flatten into 1D vector
    • x-axis is price
    • y-axis is orders. Price/time priority queue, FIFO.
      • Buy-side: ascending in price, descending in time.
      • Sell-side: ascending in price, ascending in time.
    • in other words
      • Buy-side: the higher the price and the earlier the order, the nearer we should put it to the center of the matching.
      • Sell-side: the lower the price and the earlier the order, the nearer we should put it to the center of the matching.

x-axis

line of prices

with y-axis cast into x-axis

Id   Side    Time   Qty   Price   Qty    Time   Side  
---+------+-------+-----+-------+-----+-------+------
#3                        20.30   200   09:05   SELL  
#1                        20.30   100   09:01   SELL  
#2                        20.25   100   09:03   SELL  
#5   BUY    09:08   200   20.20                       
#4   BUY    09:06   100   20.15                       
#6   BUY    09:09   200   20.15                       

Order book from Coinbase Pro

The Single Stock-Exchange Simulator

  • example 2: pro-rata

pure pro-rata

How to implement the price-time FIFO matching algorithm?

  • shard by stock, CP over AP: one stock one partition
  • stateful in-memory tree-map
    • periodically iterate the treemap to match orders
  • data persistence with cassandra
  • in/out requests of the order matching services are made through messaging queues
  • failover
    • the in-memory tree-maps are snapshotting into database
    • in an error case, recover from the snapshot and de-duplicate with cache

How to transmit data of the order book to the client-side in realtime?

  • websocket

How to support different kinds of orders?

  • same SELL or BUY: qty @ price in the treemap with different creation setup and matching conditions
    • Market Orders: place the order at the last market price.
    • Limit Orders: place the order with at a specific price.
    • Stop-Loss Orders: place the order with at a specific price, and match it in certain conditions.
    • Fill-or-Kill Orders: place the order with at a specific price, but match it only once.
    • Duration of Orders: place the order with at a specific price, but match it only in the given time span.

Orders Service

  • Preserves all active orders and order history.
  • Writes to order matching when receives a new order.
  • Receives matched orders and settle with external clearing house (async external gateway call + cronjob to sync DB)

References

I came across “bozo management” on the Blind App, and find it’s a very interesting concept. Steve Jobs coined this phrase.

If you cannot watch the video, here are the words from him.

We went through that stage at Apple where we thought, ‘Oh, we’re going to be a big company, let’s go out and hire professional management.’ We went out and hired a bunch of professional management; it didn’t work at all. Most of them were Bozos. They knew how to manage, but they didn’t know how to DO anything.

If you are a great person, why do you want to work for somebody you cannot learn anything from? And you know what’s interesting - you know what the best managers are? They are the great individual contributors who never ever wanted to be a manager, but decide they have to be a manager because no one else is able to do as good job as them.

What is DIF?

Decentralized Identity Foundation builds ecosystem for decentralized identity and ensures interop between all participants.

Why? … What is the problem?

ID Problem

The problem of decoupling ID from Personally identifiable information (PII). Identity is composed of a deeply personal collection of data that defines us, and your identity should answer to no one but you.

Specifically, challenges are

  1. Decoupling ID from identity providers. DIDs should be self-sovereign and not owned or controlled by central authorities.
  2. Decoupling ID lookup from centralized systems. DIDs and data should be able to be found across decentralized systems, so that no central owner can do the evil.
  3. Decoupling ID data from indiscreet or unknown storages. DIDs should be able to control precisely what to or what not to share with others.

Uniting the fragmented landscapes

DIF is the organization uniting the fragmented to solve the DID problem together and build an ecosystem as an industry standard.

How do they organize the efforts?

  • Designing Specs
  • Implementations
  • Aligning industry participants

People and Organizations

Working Groups

  • ID, Names, Discovery
  • Storage and Compute
  • Claims and Credentials

Members

  • Blockstack
  • Microsoft / IBM
  • HyperLedger
  • RSA
  • Ontology
  • Civic
  • iota

Ecosystem

DIF Ecosystem

DID

ID that is

  1. globally unique
  2. resolveable with high availability, and
  3. cryptographically verifiable.

DID Format: URN

Format in URN

DID methods (further explained below) define how DIDs work with a specific blockchain.

DID Document

  • DID infrastructure = a global key-value database of <DID, DID Document>
  • DID document = public keys, authentication protocols, and service endpoints for verifying the entity and explaining how to use it. It may contain three things:
    • proof purposes
    • verification methods
    • service endpoints

How DID preserves privacy?

  1. Pairwise-pseudonymous DIDs
  2. Off-chain private data
  3. Selective disclosure

DID method specification (DID <> Blockchain)

Defining how a DID and DID document are created, resolved, and managed (CRUD) on a specific blockchain.

How to join the DID method registration?

https://w3c-ccg.github.io/did-method-registry/#the-registry

Examples

Learning by Example: Blockstack DID method

  • How Blockstack leverages DID?
  • How to create a blockstack DID? How Blockstack acts as an Identity Provider?
  • How to resolve a DID?

Blockstack Naming Service (BNS)

  • Naming Layer = username <> pub key & pointer to storage
  • BNS is blockchain-agnostic. migrated from namecoint to bitcoin.
  • a Blockstack DID is defined as a pointer to the nth name registered by an address.

two categories

on-chain DIDs

  • two tx to register a name on-chain - owner address <> name
  • one owner address may have multiple on-chain names and corresponding DIDs
    • e.g. did:stack:v0:15gxXgJyT5tM5A4Cbx99nwccynHYsBouzr-3 means the fourth on-chain name was created and initially assigned to the address 15gxXgJyT5tM5A4Cbx99nwccynHYsBouzr.

off-chain DIDs. a.k.a. subdomains

  • encoded in batches, hashed, and written to a blockchain later.
  • Off-chain names are instantiated by an on-chain name, indicated by the off-chain name’s suffix. cicero.res_publica.id is processed by the owner of res_publica.id but are not owned by it.

Blockstack as an Identity Provider

Demo: https://bitpatron.co/ login with Blockstack

sequenceDiagram User ->> First Party: GET /login Note left of User: Login Note right of First Party: store transitKey First Party ->> User: Redirect User ->> Blockstack: GET /auth?authRequest=:authRequestJwt Blockstack ->> First Party: GET /manifest.json (CORS) Note left of User: create or select ID Blockstack ->> User: Redirect User ->> First Party: GET /auth?authResponse=:authResponseJwt Note right of First Party: decrypt w. transitKey Note left of First Party: pending sign-in First Party ->> Blockstack: GET /v1/names/:acctName.id.blockstack Blockstack ->> First Party: UserData
// authRequestJwt
{
  "typ": "JWT",
  "alg": "ES256K"
}
{
  "jti": "4d06f08b-67a7-4f7c-89fc-b8164b81f67a",
  "iat": 1563432343,
  "exp": 1566110743,
  "iss": "did:btc-addr:19sxvnAxPXZYAEdpF7Tti6MSVhxA8PSdCT",
  "public_keys": [
    "03994ec7b23a8e11e40684c9b2d29febf103bd92c4bbd295f1e2537042c93ac977"
  ],
  "domain_name": "http://localhost:4104",
  "manifest_uri": "http://localhost:4104/manifest.json",
  "redirect_uri": "http://localhost:4104/",
  "version": "1.3.1",
  "do_not_include_profile": true,
  "supports_hub_url": true,
  "scopes": [
    "store_write"
  ]
}
// authResponse
{
  "typ": "JWT",
  "alg": "ES256K"
}
{
  "jti": "30773b78-3595-499f-bbb3-d1e649470c70",
  "iat": 1563432894,
  "exp": 1566111294,
  "iss": "did:btc-addr:1DpKMqxBnuSSQMNun1obciPSfD9rD8KNUH",
  "private_key": "redacted - encrypted with transitKey",
  "public_keys": [
    "027c8547681cc27e27b73ee0f3c0534bdd38993dcb4c1934bf424f0b3a04dcad63"
  ],
  "profile": null,
  "username": "kirbystar.id.blockstack",
  "core_token": null,
  "email": null,
  "profile_url": "https://gaia.blockstack.org/hub/1DpKMqxBnuSSQMNun1obciPSfD9rD8KNUH/profile.json",
  "hubUrl": "https://hub.blockstack.org",
  "blockstackAPIUrl": "https://core.blockstack.org",
  "associationToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NksifQ.eyJjaGlsZFRvQXNzb2NpYXRlIjoiMDNhMTU5YzY4YWQ1ZjFkNzcxMWY2NjJmNThkNjdmMzZlNzY3ZTBjMDBhOTU4ZWY0NzljNzU3MzU0MGFkMzExZjk2IiwiaXNzIjoiMDI3Yzg1NDc2ODFjYzI3ZTI3YjczZWUwZjNjMDUzNGJkZDM4OTkzZGNiNGMxOTM0YmY0MjRmMGIzYTA0ZGNhZDYzIiwiZXhwIjoxNTk0OTY4ODk0LjcwNSwiaWF0IjoxNTYzNDMyODk0LjcwNSwic2FsdCI6IjE4NGVhMWQyMzM3MWQ1MmYyYzhmNTAyOGUwMWYxYmZiIn0.ZceaVcIK2Z8wu6KBYOHQaK7y6BI7NfxrixphOCPs1B4hZcGYDKsuf0anbm4CdAAJbKRifCm-MYHE6fjKD9E7GQ",
  "version": "1.3.1"
}
// acctName response
{
  "blockchain": "bitcoin",
  "status": "submitted_subdomain",
  "last_txid": "851ca5e6c06723e61037aa397966aafa1a6dd7159e9e31e53116106b87101886",
  "zonefile": "$ORIGIN kirbystar.id.blockstack\n$TTL 3600\n_http._tcp\tIN\tURI\t10\t1\t\"https://gaia.blockstack.org/hub/1DpKMqxBnuSSQMNun1obciPSfD9rD8KNUH/profile.json\"\n\n",
  "address": "1DpKMqxBnuSSQMNun1obciPSfD9rD8KNUH",
  "zonefile_hash": "a9c016921a9a60e04776251db53a8881e6d128ce"
}
// session
{
  "version": "1.0.0",
  "userData": {
    "username": "kirbystar.id.blockstack",
    "profile": {
      "@type": "Person",
      "@context": "http://schema.org",
      "api": {
        "gaiaHubConfig": {
          "url_prefix": "https://gaia.blockstack.org/hub/"
        },
        "gaiaHubUrl": "https://hub.blockstack.org"
      }
    },
    "email": null,
    "decentralizedID": "did:btc-addr:1DpKMqxBnuSSQMNun1obciPSfD9rD8KNUH",
    "identityAddress": "1DpKMqxBnuSSQMNun1obciPSfD9rD8KNUH",
    "appPrivateKey": "redacted",
    "coreSessionToken": null,
    "authResponseToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NksifQ.eyJqdGkiOiIyNmM4ZGM2Ny1lNzEwLTRlZDUtYmIxYi0yY2I1ODY5YTRkMTEiLCJpYXQiOjE1NjM0MzU2NjEsImV4cCI6MTU2NjExNDA2MSwiaXNzIjoiZGlkOmJ0Yy1hZGRyOjFEcEtNcXhCbnVTU1FNTnVuMW9iY2lQU2ZEOXJEOEtOVUgiLCJwcml2YXRlX2tleSI6IjdiMjI2OTc2MjIzYTIyNjQzMDMzMzMzODMyNjYzNTYxNjIzNjM0Mzg2MTM1MzYzNTM2MzI2MzMzNjEzMTYyMzY2NTYzNjEzNzMzNjU2NjIyMmMyMjY1NzA2ODY1NmQ2NTcyNjE2YzUwNGIyMjNhMjIzMDMyNjEzMzY1NjEzMzY0NjMzNjMyMzgzODY2MzgzMTY2MzAzNDMwMzU2MzY1MzI2MzMzNjUzNjY0MzMzMzMxMzQzNDM0NjY2MTM4MzA2NTY1MzIzMzMyNjEzMjM2NjQzMzMwMzczMzM2MzY2NTMzMzkzNzM3MzczMjMxMzMzNzM5MzcyMjJjMjI2MzY5NzA2ODY1NzI1NDY1Nzg3NDIyM2EyMjM2MzMzOTYzNjMzMDM5MzQzOTYyMzMzNjY0NjI2NTM1MzIzMTM0MzIzMjM3MzA2MzY1NjIzMTM0MzEzNjY2MzA2MzM0NjYzNDM4MzgzMjY0MzUzOTM0MzMzNDM2NjM2NjYyNjEzNzMxNjM2MjM1MzYzMTM2MzczMDY0MzUzMzY0MzQzNTY1MzMzODYyNjIzNzM1NjMzMjMzMzYzNjMyMzUzODMxMzgzOTM3NjYzMjMwNjMzNTM4MzA2NTMyMzEzODM0MzMzNTMwMzMzNjM1NjE2NDM1NjEzMTM4NjY2MjMyNjY2NDM3MzQ2MzY1NjMzNDM0MzI2NTY0MzY2NTY2NjYzMjM5NjIzNjY2Mzk2MTMyMzgzMTM4MzEzMzYyMzMzMDYyMzkzMTYzMzE2MzM1Mzg2MTM4MzgzMTM5MzQ2MjYzMjIyYzIyNmQ2MTYzMjIzYTIyMzkzNzY0MzM2MzM4NjEzOTYxMzQ2NjMyMzkzNjM1MzM2MzM1MzQzNzY1MzIzNTYzMzMzMzM4MzIzMDYxMzczMjYyMzU2NjY0MzQ2NjM1NjE2NDYxNjY2NjMwMzQzNzM1Mzk2NDM0MzI2MTYxMzgzNDM1MzkzNzY2MzY2MjM5NjEyMjJjMjI3NzYxNzM1Mzc0NzI2OTZlNjcyMjNhNzQ3Mjc1NjU3ZCIsInB1YmxpY19rZXlzIjpbIjAyN2M4NTQ3NjgxY2MyN2UyN2I3M2VlMGYzYzA1MzRiZGQzODk5M2RjYjRjMTkzNGJmNDI0ZjBiM2EwNGRjYWQ2MyJdLCJwcm9maWxlIjpudWxsLCJ1c2VybmFtZSI6ImtpcmJ5c3Rhci5pZC5ibG9ja3N0YWNrIiwiY29yZV90b2tlbiI6bnVsbCwiZW1haWwiOm51bGwsInByb2ZpbGVfdXJsIjoiaHR0cHM6Ly9nYWlhLmJsb2Nrc3RhY2sub3JnL2h1Yi8xRHBLTXF4Qm51U1NRTU51bjFvYmNpUFNmRDlyRDhLTlVIL3Byb2ZpbGUuanNvbiIsImh1YlVybCI6Imh0dHBzOi8vaHViLmJsb2Nrc3RhY2sub3JnIiwiYmxvY2tzdGFja0FQSVVybCI6Imh0dHBzOi8vY29yZS5ibG9ja3N0YWNrLm9yZyIsImFzc29jaWF0aW9uVG9rZW4iOiJleUowZVhBaU9pSktWMVFpTENKaGJHY2lPaUpGVXpJMU5rc2lmUS5leUpqYUdsc1pGUnZRWE56YjJOcFlYUmxJam9pTUROaE1UVTVZelk0WVdRMVpqRmtOemN4TVdZMk5qSm1OVGhrTmpkbU16WmxOelkzWlRCak1EQmhPVFU0WldZME56bGpOelUzTXpVME1HRmtNekV4WmprMklpd2lhWE56SWpvaU1ESTNZemcxTkRjMk9ERmpZekkzWlRJM1lqY3paV1V3WmpOak1EVXpOR0prWkRNNE9Ua3paR05pTkdNeE9UTTBZbVkwTWpSbU1HSXpZVEEwWkdOaFpEWXpJaXdpWlhod0lqb3hOVGswT1RjeE5qWXhMak0yT1N3aWFXRjBJam94TlRZek5ETTFOall4TGpNMk9Td2ljMkZzZENJNklqVXhOMkZsWkdVd1ltVmpOMkpqTlRnek5qY3lOREkwT1RsaE1EVm1OVEEwSW4wLjlkY2VHX3I4OVdDSUVsTklseFNtUGxPblhiSVNsZEZDejJxOTJRMnpJSk9XXzhnTjVYT0xsZnNkREJVamlQZlU3eTNyRGFXSUxfTUJicVVnVnBFanhRIiwidmVyc2lvbiI6IjEuMy4xIn0.0Xqtw-71TJ9ybWx4Uxre0Gxkisay20xn1vqwr0WaKvVeCzwv_NO6YZnVOmGPM4cF4wex06yLYWasqQWgCi-m_g",
    "hubUrl": "https://hub.blockstack.org",
    "gaiaAssociationToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NksifQ.eyJjaGlsZFRvQXNzb2NpYXRlIjoiMDNhMTU5YzY4YWQ1ZjFkNzcxMWY2NjJmNThkNjdmMzZlNzY3ZTBjMDBhOTU4ZWY0NzljNzU3MzU0MGFkMzExZjk2IiwiaXNzIjoiMDI3Yzg1NDc2ODFjYzI3ZTI3YjczZWUwZjNjMDUzNGJkZDM4OTkzZGNiNGMxOTM0YmY0MjRmMGIzYTA0ZGNhZDYzIiwiZXhwIjoxNTk0OTcxNjYxLjM2OSwiaWF0IjoxNTYzNDM1NjYxLjM2OSwic2FsdCI6IjUxN2FlZGUwYmVjN2JjNTgzNjcyNDI0OTlhMDVmNTA0In0.9dceG_r89WCIElNIlxSmPlOnXbISldFCz2q92Q2zIJOW_8gN5XOLlfsdDBUjiPfU7y3rDaWIL_MBbqUgVpEjxQ"
  },
  "transitKey": "redacted"
}

Getting Entity from DID - Universal Resolver

Domain Name <--DNS-->  IP
Repensented Entity <--Universal Resolver--> Self-sovereign Identifiers
DID <--Universal Resolver--> DID Document

Universal Resolver

Drivers for Example:

  • did:stack: DID registered from BlockStack, like did:stack:v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0
  • did:btcr: DID registered from BTC
  • etc.

Run a resolver

git clone https://github.com/decentralized-identity/universal-resolver.git
cd universal-resolver/
docker-compose -f docker-compose.yml pull
docker-compose -f docker-compose.yml up
curl -X GET http://localhost:8080/1.0/identifiers/did:stack:v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0 | jq .

{
  "redirect": null,
  "didDocument": {
    "id": "did:stack:v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0",
    "service": [
      {
        "type": "blockstack",
        "serviceEndpoint": "https://core.blockstack.org"
      }
    ],
    "publicKey": [
      {
        "id": "did:stack:v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0",
        "type": "Secp256k1VerificationKey2018",
        "publicKeyHex": "0232131c807c4b184582280bca141f2583f6a1de2e0d3e6984cdb4724527f581fa"
      }
    ],
    "@context": "https://w3id.org/did/v0.11"
  },
  "resolverMetadata": {
    "duration": 96,
    "driverId": "did-stack",
    "driver": "HttpDriver",
    "didUrl": {
      "didUrlString": "did:stack:v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0",
      "did": {
        "didString": "did:stack:v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0",
        "method": "stack",
        "methodSpecificId": "v0:SZBrgLTLXZL9ZAX8GVNgvZKcU4DJBXkUQr-0",
        "parseTree": null,
        "parseRuleCount": null
      },
      "parameters": null,
      "parametersMap": {},
      "path": "",
      "query": null,
      "fragment": null,
      "parseTree": null,
      "parseRuleCount": null
    }
  },
  "methodMetadata": {}
}

Or use Blockstack’s resolver

https://github.com/blockstack/blockstack-core/blob/master/docs/blockstack-did-spec.md#33--resolving-a-blockstack-did

DID <> Real-world: Verifiable Claims

Now we know how to recognize and resolve “who’s who” without inherently carrying personally-identifiable information. However, what if we want DID to associate with real-world entities?

Imagine that Alice has a state-issued DID and wants to buy some alcohol without disclosing her real name and precise age.

drinking

The answer is to use “verifiable claims” (aka: credentials, attestations).

  1. claim = properties we know about the entity in subject-property-value relationships, e.g. name, email, age, membership, etc.
  2. verifiable = proofs (signatures) attached

Verifiable Claims Data Model

Identity Profiletype: unordered set of URIssignature: Signature [0…1]Entity Credentialid: URItype: unordered set of URIsissuer: URIissued: date in string formclaim: ClaimClaim(at least one custom property)Verifiable Claimsignature: SignatureSignature(varies, but expected to includeat least a signature, a referenceto the signing entity, and arepresentation of the signing date)idid10…*

A simple identity profile

{
  "id": "did:ebfeb1f712ebc6f1c276e12ec21",
  "type": ["Identity", "Person"],
  "name": "Alice Bobman",
  "email": "[email protected]",
  "birthDate": "1985-12-14",
  "telephone": "12345678910"
}

A simple claim

{
  "id": "http://example.gov/credentials/3732",
  "type": ["Credential", "ProofOfAgeCredential"],
  "issuer": "https://dmv.example.gov",
  "issued": "2010-01-01",
  "claim": {
    "id": "did:ebfeb1f712ebc6f1c276e12ec21",
    "ageOver": 21
  }
}

A simple verifiable claim

{
  "@context": "https://w3id.org/security/v1",
  "id": "http://example.gov/credentials/3732",
  "type": ["Credential", "ProofOfAgeCredential"],
  "issuer": "https://dmv.example.gov",
  "issued": "2010-01-01",
  "claim": {
    "id": "did:ebfeb1f712ebc6f1c276e12ec21",
    "ageOver": 21
  },
  "revocation": {
    "id": "http://example.gov/revocations/738",
    "type": "SimpleRevocationList2017"
  },
  "signature": {
    "type": "LinkedDataSignature2015",
    "created": "2016-06-18T21:19:10Z",
    "creator": "https://example.com/jdoe/keys/1",
    "domain": "json-ld.org",
    "nonce": "598c63d6",
    "signatureValue": "BavEll0/I1zpYw8XNi1bgVg/sCneO4Jugez8RwDg/+
    MCRVpjOboDoe4SxxKjkCOvKiCHGDvc4krqi6Z1n0UfqzxGfmatCuFibcC1wps
    PRdW+gGsutPTLzvueMWmFhwYmfIFpbBu95t501+rSLHIEuujM/+PXr9Cky6Ed
    +W3JT24="
  }
}

It equals to a JOSE JWT verifiable claim

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2Rtdi
5leGFtcGxlLmdvdiIsImlhdCI6MTI2MjMwNDAwMCwiZXhwIjoxNDgzMjI4ODAwL
CJhdWQiOiJ3d3cuZXhhbXBsZS5jb20iLCJzdWIiOiJkaWQ6ZWJmZWIxZjcxMmVi
YzZmMWMyNzZlMTJlYzIxIiwiZW50aXR5Q3JlZGVudGlhbCI6eyJAY29udGV4dCI
6Imh0dHBzOi8vdzNpZC5vcmcvc2VjdXJpdHkvdjEiLCJpZCI6Imh0dHA6Ly9leG
FtcGxlLmdvdi9jcmVkZW50aWFscy8zNzMyIiwidHlwZSI6WyJDcmVkZW50aWFsI
iwiUHJvb2ZPZkFnZUNyZWRlbnRpYWwiXSwiaXNzdWVyIjoiaHR0cHM6Ly9kbXYu
ZXhhbXBsZS5nb3YiLCJpc3N1ZWQiOiIyMDEwLTAxLTAxIiwiY2xhaW0iOnsiaWQ
iOiJkaWQ6ZWJmZWIxZjcxMmViYzZmMWMyNzZlMTJlYzIxIiwiYWdlT3ZlciI6Mj
F9fX0.LwqH58NasGPeqtTxT632YznKDuxEeC59gMAe9uueb4pX_lDQd2_UyUcc6
NW1E3qxvYlps4hH_YzzTuXB_R1A9UHXq4zyiz2sMtZWyJkUL1FERclT2CypX5e1
fO4zVES_8uaNoinim6VtS76x_2VmOMQ_GcqXG3iaLGVJHCNlCu4

DID and IOT

slides

Takeaways

DID

  • is probably a Differentiator for identity-oriented vendors but a Neutralizer (out of MMRs, neutralizers, and differentiators features) for others.
  • preserves privacy in a large group of products, so that no central company could know all your stuff from all your accounts from various products.
  • combined with verifiable claims, it leverage crypto but still requires trust in the physical world.

What blockchain developers can do?

  • Join DIF.
  • Proposing a DID method on how to operate DIDs on their blockchain.
  • Building our own decentralized identity provider.

References

Answer: Amazon wanted to buy a customer for its grocery services.

Background

  • Amazon acquiring Whole Foods = Apple’s iPhone beating Palm

    • Don’t misunderstand goals vs strategies vs tactics - Apple’s strategy
      • is not to build a phone but to build personal computer
      • is not to add functionalities to a phone but to reduce the phone to an app
      • is not to duplicate the carriers but to leverage their customer connections
    • iPhone is the most successful product of all time = Amazon is the most dominant company of all time
  • Amazon’s Goal

    1. Initially, Amazon.com’s objective is to be the leading online retailer of information-based products and services, with an initial focus on books.
    2. Then, it says “our vision is to be earth’s most customer centric company; to build a place where people can come to find and discover anything they might want to buy online.”
    3. Amazon’s goal is to take a cut of all economic activity.
  • Amazon’s Strategy

    • to enterprise: AWS. Assuming that all businesses will soon be Internet-enabled businesses.
    • to customer: Prime. Assuming that superior cost/and/superior selection are not sustainable. With prime, alternatives won’t be even considered by customers.
      • However
        • grocery is the largest retail category.
        • grocery is the most persistent opportunity for reminding users there are other alternatives.
  • Tactics: develop grocery services

Why hadn’t Amazon figured out the right tactics?

Book Grocery
high SKUs = large selection less SKUs(30k - 50k)
standardized vary in quality
imperishable perishable

AmazonFresh’s cost disadvantage

  1. High costs of perishable items if not scaled
  2. Scale needs to be based on cities

Why can acquiring Whole Food (not doing other things) solve the scale problem?

Primitives model for business with 1) hight fixed costs 2) high returns to scale

  • decouple infrastructure into Minimal Sellable Units (MSUs)
  • business itself is The First-And-Best Customer of those MSUs
  • resell MSUs to the outside

AWS Three Layers

Services Primitives S3, EC2, RDS, SNS, …
Platform AWS High Fixed Costs + Returns to Scale
Infrastructure Modularized Components Data center, Servers, Storage, Switches, Bandwidth
  • MSUs are S3, EC2, RDS, SNS, etc
  • The First-And-Best Customer is amazon.com
  • resell MSUs to non-Amazon developers

Amazon.com Three Layers

Services Packages FDA, Amazon Pay, …
Platform Fulfillment Centers High Fixed Costs + Returns to Scale
Infrastructure Modularized Suppliers Manufacturers, 3rd Party Merchants, …
  • MSUs are FDA, Amazon Pay, etc.
  • The First-And-Best Customer is Amazon first-party e-commerse
  • resell MSUs to 3rd party merchants

The insight here is that grocery business has no first-and-best customer.

Perfect Customer

After fitting in Whole Foods to the big picture, we can see that Amazon is buying more than a retailer - it’s buying a customer.

Amazon.com Three Layers + Customers

Customers Whole Foods, Delivery, Restaurants
Services Groceries Meat, Fruit, Vegetables, Non-perishables, …
Platform Fulfillment Centers High Fixed Costs + Returns to Scale
Infrastructure Modularized Suppliers Store Brand, Name Brand, Local Suppliers, Regional Suppliers, …

Now Amazon Grocery Services can serve AmazonFresh and WholeFoods, and then in the future restaurants or whatever can consume it.

TianPan.co

Startup Engineering
© 2010-2018 Tian
Built with in San Francisco