Bitcoin wallet and interface design — key management, transaction construction, and RPC

Figures: Satoshi Nakamoto

Introduction

This page is L1 #8 — Wallet design in the design-document series. It covers the application layer of Bitcoin Core: how the software manages keys, derives addresses, selects coins, constructs and signs transactions, estimates fees, and exposes its functionality to external callers. It depends on the transaction design page for the UTXO model and script types, and on the cryptography design page for key derivation and signature primitives.

The wallet is the only subsystem in Bitcoin that touches private keys. Every other component — validation, relay, storage, consensus — operates exclusively on public data. This asymmetry has driven the wallet’s evolution from a tightly coupled module inside the node binary to a logically (and increasingly physically) separated process.

Where behavior differs between the Satoshi-era implementation (v0.1, January 2009) and modern Bitcoin Core (v27+ baseline), both are noted.

1. Wallet architecture overview

The diagram below shows the wallet’s position within the Bitcoin Core process and the interfaces it communicates through. In v27+ the wallet can optionally run as a separate process connected to the node via IPC.

Wallet (same or separate process)

Bitcoin Core node process

External callers

JSON-RPC

JSON-RPC / REST

HTTP GET

subscribe

mempool snapshot

new block

broadcast tx

query UTXOs

fee estimates

new block

bitcoin-cli

Third-party application

ZMQ subscriber

RPC server

(JSON-RPC over HTTP)

REST interface

(read-only, no auth)

ZMQ publisher

(push notifications)

Validation engine

Mempool

P2P network layer

Chain state

(UTXO set)

Key manager

(descriptors + keypool)

Coin selection

Transaction builder

Signer

(ECDSA / Schnorr)

Fee estimator

Wallet database

(SQLite, v27+)

Satoshi-era architecture

In v0.1, the wallet was embedded in the node binary with no interface boundary. The GUI, the key store, the miner, and the validation engine all shared a single process and a single Berkeley DB file (wallet.dat). There was no RPC server at initial release; JSON-RPC was added within the first weeks of operation.

2. Key management

Key management is the wallet’s most critical responsibility. Losing a private key means losing the funds it controls — permanently, with no recovery authority to appeal to.

Legacy key pool (v0.1–v0.20)

Satoshi’s v0.1 wallet generated keys independently: each private key was a fresh random number stored in wallet.dat. A pre-generated pool of 100 keys (the “keypool”) provided lookahead so that a backup taken today could cover addresses generated in the near future. But any key created after the last backup was lost if the wallet file was destroyed.

Descriptor wallets (v0.21+, default in v23+)

Modern Bitcoin Core replaces the legacy key model with output descriptors (BIP 380–386): composable expressions that fully specify how the wallet derives addresses and recognizes its own outputs.

Raw seed entropy

(BIP 32 master seed)

Descriptor:

wpkh([fp/84h/0h/0h]xpub.../0/*)

Descriptor:

tr([fp/86h/0h/0h]xpub.../0/*)

Descriptor:

pkh([fp/44h/0h/0h]xpub.../0/*)

Derived keypool

(P2WPKH addresses:

bc1q...)

Derived keypool

(P2TR addresses:

bc1p...)

Derived keypool

(P2PKH addresses:

1...)

SQLite wallet database

PropertyLegacy walletDescriptor wallet (v27+)
Key sourceIndependent random keysDeterministic derivation from descriptors
Backup modelExport wallet.dat after every new keyOne-time seed backup covers all future keys
Script type awarenessWallet infers type from key contextDescriptor explicitly declares the script type
Watch-only supportSeparate watch-only wallet importAny descriptor with an xpub (no private key) becomes watch-only
Multi-script supportImplicit (BIP 44/49/84/86 conventions)Explicit — each descriptor is a distinct derivation rule
Storage backendBerkeley DB (wallet.dat)SQLite (wallet.dat in new format)
Migration pathmigratewallet RPC converts legacy → descriptor

3. Address generation

A Bitcoin address is a user-facing encoding of a locking-script payload. The wallet generates addresses from its active descriptors, cycling through the keypool as addresses are handed out.

P2PKH

P2WPKH

P2TR

Active descriptor

e.g. wpkh(xpub.../0/*)

Next unused index

(e.g. index 7)

BIP 32 child key

derivation at index 7

Public key

(compressed, 33 bytes)

Hash160

→ 20-byte hash

Base58Check

→ 1...

Hash160

→ 20-byte hash

Bech32

→ bc1q...

Taproot tweak

(x-only key)

Bech32m

→ bc1p...

The wallet maintains a gap limit — the number of consecutive unused addresses it derives ahead of the last used one. When a transaction is detected that uses address at index n, the wallet extends the keypool so that there are always unused addresses available. This ensures that restoring from a seed phrase can rediscover all funded addresses by scanning forward until the gap limit is reached without finding any transactions.

The cryptographic details of key derivation (BIP 32 HMAC-SHA512, hardened vs normal paths, secp256k1 scalar multiplication) and the encoding formats (Base58Check, Bech32, Bech32m) are covered in the cryptography design page.

4. Coin selection

When a wallet builds a transaction, it must choose which UTXOs to spend as inputs. The coin selection problem sits at the intersection of fee optimization, privacy, and UTXO pool health. The transaction design page introduces the strategies; this section details the modern selection pipeline.

Selection pipeline (v27+)

exact match found

no exact match

Payment request

(amount + fee rate)

Filter eligible UTXOs

(confirmed, not locked,

above dust threshold)

Branch-and-bound search

(exact match, no change)

Score with waste metric

Knapsack selection

(random trials,

minimize excess)

Score with waste metric

Single random draw

(shuffle and accumulate

until target met)

Score with waste metric

Pick lowest-waste

candidate

Build transaction

(inputs + outputs +

change if needed)

Waste metric

The waste metric scores each candidate selection by comparing the cost of spending the chosen UTXOs now versus spending them at a hypothetical long-term fee rate. It penalizes unnecessary change outputs and rewards exact-match (changeless) solutions. The formula balances three components:

ComponentWhat it measuresEffect
Input costWeight of selected inputs × (current fee rate − long-term fee rate)Positive when overpaying relative to the long-term rate; negative when current fees are low (good time to consolidate)
Change costCost to create and later spend the change outputAlways positive — change is an additional output now and an additional input later
ExcessValue above the target that would be lost as fee in a changeless transactionApplies only to BnB solutions where the slight overshoot is donated to fees instead of creating change

A lower waste score is better. The wallet evaluates all candidate selections from different strategies and picks the one with the lowest waste.

5. PSBT workflow

Partially Signed Bitcoin Transactions (BIP 174 / BIP 370) separate transaction construction from signing. A PSBT carries all the metadata a signer needs — UTXO data, derivation paths, redeem scripts — so that signing can happen on a device that has no access to the blockchain.

NodeFinalizerSigner 2(hardware device)Signer 1(hot wallet)UpdaterCreatorNodeFinalizerSigner 2(hardware device)Signer 1(hot wallet)UpdaterCreatorCreate unsigned PSBT(inputs, outputs, no signatures)Pass PSBTAdd UTXO data,derivation paths,sighash typesPass updated PSBTSign own inputs(add partial signatures)Pass partially signed PSBTVerify outputs on screen,sign remaining inputsPass fully signed PSBTCombine partial signatures,assemble final scriptSig/witness,produce network-ready transactionBroadcast raw transactionValidate and relay

PSBT roles

RoleWhat it doesWho typically fills it
CreatorDefines inputs and outputs; produces an unsigned PSBTWallet software, createpsbt RPC
UpdaterAttaches UTXO details, derivation info, scriptsSame wallet or a coordinator; walletprocesspsbt RPC
SignerAdds a partial signature for the inputs it controlsHardware wallet, air-gapped machine, or walletprocesspsbt
CombinerMerges partial signatures from multiple signers into one PSBTcombinepsbt RPC
FinalizerAssembles the final scriptSig / witness from all signaturesfinalizepsbt RPC
ExtractorPulls the complete, network-ready transaction from the finalized PSBTfinalizepsbt (returns both PSBT and raw hex)

Why PSBT matters. Before BIP 174, multi-device signing required ad-hoc formats and vendor-specific protocols. PSBT provides a single, portable, self-describing container that any compliant wallet can produce, update, sign, and finalize. It is the foundation for hardware-wallet integration, multisig workflows, CoinJoin coordination, and any scenario where the device constructing the transaction is not the device holding the keys.

6. Fee estimation

Bitcoin transactions compete for limited block space. The wallet must estimate what fee rate (satoshis per virtual byte) is likely to confirm a transaction within a target number of blocks.

Estimation mechanism

Bitcoin Core’s estimatesmartfee tracks the fee rates of transactions as they enter the mempool and records how many blocks each waited before confirmation. Transactions are bucketed by fee rate, and for each bucket the estimator maintains a success rate: what fraction of transactions at that fee rate confirmed within n blocks. The estimate for a given confirmation target is the lowest fee-rate bucket whose success rate exceeds a threshold (85% for economical mode, 95% for conservative mode).

ModeThresholdBehaviorTypical use
Economical85% success rateLower fees, slightly longer confirmation riskNon-urgent payments
Conservative95% success rateHigher fees, more reliable confirmationTime-sensitive payments

Fee estimation evolution

AspectSatoshi era (v0.1)Modern Bitcoin Core, v27+ baseline
Fee requirementTransactions were free; fee was optionalMandatory; unpaid transactions are rejected by mempool policy
EstimationNone — no fee market existedBucket-based mempool tracking (estimatesmartfee)
Fee bumpingNot availableRBF (bumpfee RPC), CPFP (spend the low-fee output with a high-fee child)
Minimum relay fee0.01 BTC (later reduced)1 sat/vB default (minrelaytxfee)

7. RPC, REST, and ZMQ interfaces

Bitcoin Core exposes three interfaces for external callers. Each serves a different access pattern.

Interfaces

Callers

bitcoin-cli

(command line)

Web application

Streaming consumer

(indexer, monitor)

JSON-RPC

(port 8332)

Authenticated,

read + write

REST

(port 8332)

Unauthenticated,

read-only

ZMQ

(configurable port)

Push notifications,

no request needed

RPC command categories

CategoryExample commandsWhat they do
Blockchaingetblockchaininfo, getblock, gettxoutsetinfoQuery chain state, block data, UTXO set statistics
Walletgetnewaddress, sendtoaddress, listunspent, walletprocesspsbtKey management, transaction construction, coin selection
Transactiongetrawtransaction, decoderawtransaction, sendrawtransactionInspect, create, and broadcast raw transactions
Mininggetblocktemplate, submitblock, getmininginfoBlock template construction and submission (pool operators)
Networkgetpeerinfo, addnode, getnetworkinfoPeer management, connection status, bandwidth stats
Controlstop, uptime, logging, getmemoryinfoNode lifecycle and diagnostics
FeeestimatesmartfeeFee-rate estimation for a target confirmation window
PSBTcreatepsbt, walletprocesspsbt, combinepsbt, finalizepsbtFull PSBT workflow without touching private keys directly

Interface comparison

PropertyJSON-RPCRESTZMQ
ProtocolHTTP POST (JSON body)HTTP GET (URL path)TCP push (pub/sub)
AuthenticationCookie or username/password (rpcauth)None (read-only by design)None (local network assumed)
DirectionRequest → responseRequest → responseServer pushes to subscribers
Write accessYes (send transactions, manage wallet)NoNo (notification only)
Data formatJSONJSON, binary, or hex (caller’s choice)Raw binary (block hash, tx hash, or full serialized block/tx)
Typical callerbitcoin-cli, wallet apps, exchange backendsBlock explorers, lightweight monitorsIndexers, real-time dashboards, Lightning nodes
ZMQ topicshashblock, hashtx, rawblock, rawtx, sequence

8. Wallet separation from node

Satoshi’s v0.1 compiled the wallet, the miner, the GUI, and the validation engine into a single binary. Modern Bitcoin Core has progressively decoupled the wallet from the node.

v27+ baseline

IPC

(Cap'n Proto)

bitcoind

(node only)

bitcoin-wallet

(optional separate process)

wallet.dat

(SQLite)

v0.17–v0.20

bitcoind

(node + wallet module)

Wallet module

(internal, linked library)

wallet.dat

(Berkeley DB)

v0.1 (2009)

Single binary

(wallet + node + miner + GUI)

wallet.dat

(Berkeley DB)

MilestoneVersionChange
Wallet interface definedv0.17 (2018)Internal interfaces::Wallet abstraction separates wallet logic from node logic
-disablewallet optionv0.8 (2013)Node can run without loading any wallet
Multiple wallet supportv0.17 (2018)A single node loads and manages multiple wallet files simultaneously
Descriptor walletsv0.21 (2021)New wallet format; default for new wallets from v23
Berkeley DB deprecatedv26 (2023)New wallets use SQLite exclusively; legacy BDB wallets can be migrated but BDB support is not yet fully removed
Multiprocess work (experimental)v27+ (2024)Experimental bitcoin-node / bitcoin-wallet process separation via IPC (Cap’n Proto). The bitcoin-wallet CLI tool handles offline wallet operations; full IPC-based runtime separation is in progress, not yet the default

Why separation matters. The wallet handles private keys — the most sensitive data in the system. Logical separation (already achieved) and eventual physical process separation allow the key-holding component to operate in a restricted environment while the node handles publicly verifiable chain data.

9. Two-era comparison

FeatureSatoshi era (v0.1, Jan 2009)Modern Bitcoin Core, v27+ baseline
ArchitectureWallet embedded in single-binary nodeLogically separated; experimental multiprocess separation in progress
Key generationRandom key pool (100 independent keys)Descriptor wallets: deterministic derivation from master seed
Key storageBerkeley DB (wallet.dat)SQLite (wallet.dat in new format)
Backup modelExport file after every new key; new keys after backup are unrecoverableDescriptor backup covers all derived keys (raw BIP 32 seed, not BIP 39)
Address typesP2PK, P2PKH onlyP2PKH, P2SH, P2WPKH, P2WSH, P2TR
Address encodingBase58CheckBase58Check (legacy), Bech32, Bech32m
Coin selectionSimple largest-firstBnB + knapsack + single-random-draw; waste metric scoring
Transaction signingInternal, same processInternal, PSBT (BIP 174/370), or hardware wallet via HWI
Multi-device signingNot supportedPSBT workflow: create → update → sign → combine → finalize
Fee estimationNone (most transactions were free)Bucket-based mempool tracking; economical and conservative modes
Fee bumpingNot availableReplace-by-Fee (bumpfee), CPFP
InterfacesNone at launch; basic JSON-RPC added shortly afterJSON-RPC (full), REST (read-only), ZMQ (push notifications)
Multi-walletOne wallet per nodeMultiple wallets loaded simultaneously; loadwallet / unloadwallet
EncryptionNot availableencryptwallet — AES-256-CBC encryption of private keys at rest
Watch-onlyNot supportedAny descriptor with xpub only (no private key)
Process modelMonolithic (wallet + node + miner + GUI)Modular: bitcoind (node), bitcoin-wallet (wallet), bitcoin-qt (GUI)

10. Limits of this page

This page covers the wallet and interface layer of Bitcoin Core. The following topics are out of scope and addressed in their respective pages within the design-document series:

  • Transaction structure and Script — how inputs, outputs, and locking scripts work at the protocol level. Covered in the transaction design page.
  • Cryptographic primitives — elliptic-curve math, hash functions, HD derivation internals (BIP 32 HMAC-SHA512), and signature algorithms. Covered in the cryptography design page.
  • Mempool policy — relay rules, fee-rate floors, package relay, and Replace-by-Fee enforcement. Covered in the network design page.
  • Mining and block template — how miners select transactions from the mempool and construct candidate blocks. Covered in the consensus design page.
  • Third-party wallets — Electrum, Sparrow, hardware wallets, and mobile wallets are application-layer software outside the Bitcoin Core reference implementation.
  • Lightning Network — layer-2 payment channels that build on the transaction and PSBT primitives described here but operate under a separate protocol.