# Data persistence

> For the complete documentation index, see [llms.txt](/llms.txt)

[Canister](../../concepts/canisters.md) state lives in two places: **heap memory** and **stable memory** (persistent, survives upgrades). In Rust and most languages, heap memory is wiped on upgrade: any data you care about must be stored in stable memory. In Motoko, the `persistent actor` pattern automatically preserves all actor state across upgrades without any additional work.

This guide shows how to store data durably in both Motoko and Rust. For a conceptual explanation of why stable memory works this way, see [Orthogonal Persistence](../../concepts/orthogonal-persistence.md).

## Store data durably

### Motoko

Use `persistent actor`. All `let` and `var` declarations inside the actor body are automatically persisted across upgrades. No `stable` keyword, no upgrade hooks.

```motoko
import Map "mo:core/Map";
import Nat "mo:core/Nat";
import Text "mo:core/Text";
import Time "mo:core/Time";

persistent actor {

  // Custom type: defined inside the actor body
  type User = {
    id : Nat;
    name : Text;
    created : Int;
  };

  // Automatically persisted across upgrades: no "stable" keyword needed
  let users = Map.empty<Nat, User>();
  var userCounter : Nat = 0;

  // Transient data: resets to 0 on every upgrade
  transient var requestCount : Nat = 0;

  public func addUser(name : Text) : async Nat {
    let id = userCounter;
    Map.add(users, Nat.compare, id, {
      id;
      name;
      created = Time.now();
    });
    userCounter += 1;
    requestCount += 1;
    id
  };

  public query func getUser(id : Nat) : async ?User {
    Map.get(users, Nat.compare, id)
  };

  public query func getUserCount() : async Nat {
    Map.size(users)
  };

  // Resets to 0 after every upgrade: use transient for ephemeral state
  public query func getRequestCount() : async Nat {
    requestCount
  };
}
```

**Key rules:**

- `let` for collections (`Map`, `List`, `Set`): auto-persisted, no serialization needed
- `var` for simple values (`Nat`, `Text`, `Bool`): auto-persisted
- `transient var` for caches or counters that should reset on upgrade
- No `pre_upgrade` / `post_upgrade` hooks needed. The runtime handles persistence
- Do not write `stable let` or `stable var`: redundant in `persistent actor` and produces compiler warnings

**mops.toml:**

```toml
[package]
name = "my-project"
version = "0.1.0"

[dependencies]
core = "2.0.0"
```

### Rust

Rust canisters use [`ic-stable-structures`](https://docs.rs/ic-stable-structures/latest/ic_stable_structures/) for persistent storage. The `MemoryManager` partitions stable memory into virtual memories, each backing a separate data structure. Data lives in stable memory from the start. No serialization on upgrade.

**Cargo.toml:**

```toml
[package]
name = "stable_memory_backend"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
ic-cdk = "0.19"
ic-stable-structures = "0.7"
candid = "0.10"
serde = { version = "1", features = ["derive"] }
ciborium = "0.2"
```

**Implementing Storable for custom types:**

`StableBTreeMap` keys must implement `Storable + Ord`, values must implement `Storable`. Primitive types (`u64`, `bool`, `String`, `Vec<u8>`, `Principal`) already implement `Storable`. For custom structs, implement it manually using CBOR serialization:

```rust
use ic_stable_structures::storable::{Bound, Storable};
use candid::CandidType;
use serde::{Deserialize, Serialize};
use std::borrow::Cow;

#[derive(CandidType, Serialize, Deserialize, Clone)]
struct User {
    id: u64,
    name: String,
    created: u64,
}

impl Storable for User {
    // Prefer Unbounded: avoids breakage when adding new fields.
    // Bounded requires a fixed max_size; if the encoded size of a value
    // exceeds max_size after a schema change, writes will trap.
    // Existing stored data is unaffected, but no new or updated records
    // can be written until the type fits within the declared max_size.
    const BOUND: Bound = Bound::Unbounded;

    fn to_bytes(&self) -> Cow<'_, [u8]> {
        let mut buf = vec![];
        ciborium::into_writer(self, &mut buf).expect("Failed to encode User");
        Cow::Owned(buf)
    }

    fn from_bytes(bytes: Cow<'_, [u8]>) -> Self {
        ciborium::from_reader(bytes.as_ref()).expect("Failed to decode User")
    }
}
```

**MemoryManager and stable structures:**

```rust
use ic_stable_structures::{
    memory_manager::{MemoryId, MemoryManager, VirtualMemory},
    storable::{Bound, Storable},
    DefaultMemoryImpl, StableBTreeMap, StableCell,
};
use ic_cdk::{init, post_upgrade, query, update};
use candid::CandidType;
use serde::{Deserialize, Serialize};
use std::borrow::Cow;
use std::cell::RefCell;

type Memory = VirtualMemory<DefaultMemoryImpl>;

// Each structure gets its own MemoryId: NEVER reuse IDs across structures
const USERS_MEM_ID: MemoryId = MemoryId::new(0);
const COUNTER_MEM_ID: MemoryId = MemoryId::new(1);

thread_local! {
    static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
        RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));

    static USERS: RefCell<StableBTreeMap<u64, User, Memory>> =
        RefCell::new(StableBTreeMap::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(USERS_MEM_ID))
        ));

    // StableCell for a single value (counter, config, etc.)
    static COUNTER: RefCell<StableCell<u64, Memory>> =
        RefCell::new(StableCell::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(COUNTER_MEM_ID)),
            0u64,
        ).expect("Failed to init counter"));
}

#[init]
fn init() {
    // One-time initialization: stable structures auto-initialize from above
}

#[post_upgrade]
fn post_upgrade() {
    // Stable structures auto-restore: no deserialization needed here.
    // Re-initialize timers or other transient state if needed.
}

#[update]
fn add_user(name: String) -> u64 {
    let id = COUNTER.with(|c| {
        let mut cell = c.borrow_mut();
        let current = *cell.get();
        cell.set(current + 1).expect("Counter update failed");
        current
    });

    USERS.with(|users| {
        users.borrow_mut().insert(id, User {
            id,
            name,
            created: ic_cdk::api::time(),
        });
    });

    id
}

#[query]
fn get_user(id: u64) -> Option<User> {
    USERS.with(|users| users.borrow().get(&id))
}

#[query]
fn get_user_count() -> u64 {
    USERS.with(|users| users.borrow().len())
}

ic_cdk::export_candid!();
```

**Key rules:**

- Each structure gets a unique `MemoryId`: reusing IDs corrupts both structures
- `StableBTreeMap` for keyed collections; keys need `Storable + Ord`
- `StableCell` for single values (counters, config flags)
- `StableLog` for append-only logs: requires two `MemoryId`s (index + data)
- `thread_local! { RefCell<StableBTreeMap<...>> }` is the correct pattern: `RefCell` wraps the stable structure, not a heap `HashMap`
- No `pre_upgrade`/`post_upgrade` serialization needed: data is already in stable memory

## Schema evolution

### Motoko

When upgrading a Motoko canister, the type of every persistent field must be compatible with its stored value. Violating this causes the upgrade to trap. The canister continues running on the old Wasm with its data intact, but cannot be upgraded until the type conflict is resolved.

**Safe changes (always OK):**
- Add new `let` or `var` fields with initial values
- Add new optional record fields (e.g., change `{ name : Text }` to `{ name : Text; email : ?Text }`)

**Unsafe changes (will trap on upgrade):**
- Remove or rename a persistent field
- Change a field's type (e.g., `Nat` → `Int`)
- Change a non-optional field to a different type

### Rust

When using more than one stable structure, give each a unique `MemoryId`. `StableLog` requires two memory regions (index + data).

This example extends the snippet above: it reuses the same `Memory` type alias, `MemoryManager`, `DefaultMemoryImpl`, `RefCell`, and `User` struct, and adds `Post` and `AUDIT_LOG`:

```rust
use ic_stable_structures::{
    memory_manager::{MemoryId, MemoryManager, VirtualMemory},
    DefaultMemoryImpl, StableBTreeMap, StableCell, StableLog,
};
use candid::CandidType;
use serde::{Deserialize, Serialize};
use std::cell::RefCell;

type Memory = VirtualMemory<DefaultMemoryImpl>;

#[derive(CandidType, Serialize, Deserialize, Clone)]
struct Post {
    id: u64,
    content: String,
}

// Assign one MemoryId per structure: never reuse
const USERS_MEM_ID: MemoryId = MemoryId::new(0);
const POSTS_MEM_ID: MemoryId = MemoryId::new(1);
const COUNTER_MEM_ID: MemoryId = MemoryId::new(2);
const LOG_INDEX_MEM_ID: MemoryId = MemoryId::new(3); // StableLog needs two
const LOG_DATA_MEM_ID: MemoryId = MemoryId::new(4);

thread_local! {
    static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
        RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));

    static USERS: RefCell<StableBTreeMap<u64, User, Memory>> =
        RefCell::new(StableBTreeMap::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(USERS_MEM_ID))
        ));

    static POSTS: RefCell<StableBTreeMap<u64, Post, Memory>> =
        RefCell::new(StableBTreeMap::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(POSTS_MEM_ID))
        ));

    static COUNTER: RefCell<StableCell<u64, Memory>> =
        RefCell::new(StableCell::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(COUNTER_MEM_ID)),
            0u64,
        ).expect("Failed to init counter"));

    static AUDIT_LOG: RefCell<StableLog<Vec<u8>, Memory, Memory>> =
        RefCell::new(StableLog::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(LOG_INDEX_MEM_ID)),
            MEMORY_MANAGER.with(|m| m.borrow().get(LOG_DATA_MEM_ID)),
        ).expect("Failed to init audit log"));
}
```

**Anti-pattern: pre_upgrade serialization**

Avoid serializing heap data to stable memory in `pre_upgrade` hooks. This pattern is fragile and will brick the canister under load:

```rust
// DO NOT DO THIS
#[pre_upgrade]
fn pre_upgrade() {
    // If STATE is large, this hits the instruction limit and traps.
    // A trapped pre_upgrade prevents the upgrade from completing:
    // the canister is stuck on the old code. Recovery is possible via
    // the skip_pre_upgrade flag (which bypasses the hook at the cost of
    // losing any state it would have serialized), but it's an emergency
    // measure. Avoid this pattern entirely.
    let state = STATE.with(|s| s.borrow().clone());
    ic_cdk::storage::stable_save((state,)).unwrap();
}

#[post_upgrade]
fn post_upgrade() {
    let (state,) = ic_cdk::storage::stable_restore().unwrap();
    STATE.with(|s| *s.borrow_mut() = state);
}
```

Use `StableBTreeMap` and other stable structures instead. Data lives in stable memory from the start, so no serialization step is needed on upgrade.

## Idempotency for safe data mutation

When an update call's result is unknown (network interruption, ingress expiry), callers may retry. Without idempotency, retries can cause double-writes, double-spends, or duplicate records. Two patterns handle this:

### Sequence numbers

Track a per-caller counter. A call is only accepted if it carries the next expected sequence number:

#### Motoko

```motoko
import Map "mo:core/Map";
import Nat "mo:core/Nat";
import Principal "mo:core/Principal";

persistent actor {

var callerSeq = Map.empty<Principal, Nat>();

public shared(msg) func transferWithSeq(amount : Nat, seq : Nat) : async Bool {
  let caller = msg.caller;
  let expected = switch (Map.get(callerSeq, Principal.compare, caller)) {
    case null 0;
    case (?n) n;
  };
  if (seq != expected) return false; // reject out-of-order or duplicate calls
  // ... perform transfer ...
  Map.add(callerSeq, Principal.compare, caller, seq + 1);
  true
};

}
```

#### Rust

```rust
use ic_stable_structures::{StableBTreeMap, memory_manager::{MemoryId, MemoryManager, VirtualMemory}, DefaultMemoryImpl};
use ic_cdk::{caller, update};
use candid::Principal;
use std::cell::RefCell;

type Memory = VirtualMemory<DefaultMemoryImpl>;

thread_local! {
    static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
        RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));

    static CALLER_SEQ: RefCell<StableBTreeMap<Principal, u64, Memory>> =
        RefCell::new(StableBTreeMap::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(MemoryId::new(10)))
        ));
}

#[update]
fn transfer_with_seq(amount: u64, seq: u64) -> bool {
    let caller = caller();
    CALLER_SEQ.with(|s| {
        let mut map = s.borrow_mut();
        let expected = map.get(&caller).unwrap_or(0);
        if seq != expected {
            return false; // reject out-of-order or duplicate calls
        }
        // ... perform transfer ...
        map.insert(caller, seq + 1);
        true
    })
}
```

Best for low-throughput, per-account flows (similar to Ethereum nonces). Limits concurrency to one in-flight call per caller.

### ID deduplication

Callers attach a unique ID per operation. The canister rejects duplicates within a time window:

#### Motoko

```motoko
import Map "mo:core/Map";
import Text "mo:core/Text";
import Time "mo:core/Time";

persistent actor {

type DedupeEntry = { executed_at : Int };
let executed = Map.empty<Text, DedupeEntry>();
let WINDOW_NS : Int = 24 * 60 * 60 * 1_000_000_000; // 24 hours in nanoseconds

public func transferWithId(amount : Nat, idempotency_key : Text) : async Bool {
  let now = Time.now();
  switch (Map.get(executed, Text.compare, idempotency_key)) {
    case (?entry) {
      if (now - entry.executed_at < WINDOW_NS) return true; // already done
    };
    case null {};
  };
  // ... perform transfer ...
  Map.add(executed, Text.compare, idempotency_key, { executed_at = now });
  true
};

}
```

#### Rust

```rust
use ic_stable_structures::{StableBTreeMap, memory_manager::{MemoryId, MemoryManager, VirtualMemory}, DefaultMemoryImpl};
use ic_stable_structures::storable::{Bound, Storable};
use ic_cdk::{api::time, update};
use std::borrow::Cow;
use std::cell::RefCell;

type Memory = VirtualMemory<DefaultMemoryImpl>;

const WINDOW_NS: u64 = 24 * 60 * 60 * 1_000_000_000; // 24 hours in nanoseconds

thread_local! {
    static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
        RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));

    static EXECUTED: RefCell<StableBTreeMap<String, u64, Memory>> =
        RefCell::new(StableBTreeMap::init(
            MEMORY_MANAGER.with(|m| m.borrow().get(MemoryId::new(11)))
        ));
}

#[update]
fn transfer_with_id(amount: u64, idempotency_key: String) -> bool {
    let now = time();
    EXECUTED.with(|s| {
        let mut map = s.borrow_mut();
        if let Some(executed_at) = map.get(&idempotency_key) {
            if now - executed_at < WINDOW_NS {
                return true; // already done
            }
        }
        // ... perform transfer ...
        map.insert(idempotency_key, now);
        true
    })
}
```

Supports higher throughput and concurrent callers. Requires bounded storage: expire entries after the deduplication window.

## Verify persistence across upgrades

The definitive test: deploy, write data, upgrade, confirm data survived.

### Motoko

Method names are camelCase in Motoko:

```bash
icp network start -d
icp deploy backend

# Write some data
icp canister call backend addUser '("Alice")'
icp canister call backend addUser '("Bob")'

# Record the count
icp canister call backend getUserCount '()'
# Returns: (2 : nat)

# Upgrade the canister (redeploy with code change)
icp deploy backend

# Data must still be there
icp canister call backend getUserCount '()'
# Must still return: (2 : nat)

icp canister call backend getUser '(0)'
# Returns: (opt record { id = 0 : nat; name = "Alice"; ... })

# Transient state resets
icp canister call backend getRequestCount '()'
# Returns: (0 : nat): expected, transient var resets on upgrade
```

### Rust

Method names are snake_case in Rust:

```bash
icp network start -d
icp deploy backend

# Write some data
icp canister call backend add_user '("Alice")'
icp canister call backend add_user '("Bob")'

# Record the count
icp canister call backend get_user_count '()'
# Returns: (2 : nat64)

# Upgrade the canister (redeploy with code change)
icp deploy backend

# Data must still be there
icp canister call backend get_user_count '()'
# Must still return: (2 : nat64)

icp canister call backend get_user '(0 : nat64)'
# Returns: (opt record { id = 0 : nat64; name = "Alice"; created = ... })
```

If the count drops to 0 after upgrade, the data is not in stable memory. Review your storage declarations.

## Storage recommendations

### Choose the right storage type

| Memory type | Max size | Persists across upgrades | Best for |
|-------------|----------|--------------------------|----------|
| Heap | 4 GiB | No (Rust) / Yes (Motoko `persistent actor`) | Frequently accessed data, caches, ephemeral computation |
| Stable | 500 GiB | Yes | All important data, large datasets, anything that must survive upgrades |

The practical rule: **use stable structures directly for any data that matters**. Avoid relying on `pre_upgrade` / `post_upgrade` hooks to serialize heap data to stable memory. Serializing large heap state during an upgrade can hit the instruction limit and trap, leaving the canister on the old code. Data in stable structures is already in stable memory from the first write — no serialization step required on upgrade.

For Motoko, `persistent actor` makes all `let` and `var` declarations persistent automatically. There is no need to choose manually between heap and stable memory.

### Language-specific recommendations

#### Motoko

**Choose efficient data structures.**

The `mo:core` library provides stable-friendly, performant data structures. Use these in preference to the legacy `mo:base` equivalents:

| Use case | `mo:core` type | Replaces (`mo:base`) |
|----------|----------------|----------------------|
| Key-value map | `Map` | `HashMap`, `TrieMap`, `Trie`, `RBTree` |
| Dynamic sequence | `List` | `Buffer` |
| Double-ended queue | `Queue` | `Deque` |
| Ordered map | `pure/Map` | `OrderedMap` |
| Ordered set | `pure/Set` | `OrderedSet` |

`Map` avoids the automatic resizing overhead that `HashMap` incurs on growth. `List` handles dynamic sequences without the fragile array-copy pattern of `Buffer`.

**Prefer `Blob` over `[Nat8]` for binary data.**

`Blob` is 4× more compact than `[Nat8]` and produces significantly less GC pressure. Use `Blob` for binary assets, cryptographic values, and anywhere you would send or receive `vec nat8` in Candid. Store large `Blob`s in stable memory.

**Use `compacting-gc` for append-only workloads (classical persistence only).**

If your canister grows the heap by appending data without frequent deletions, the `--compacting-gc` flag allows the GC to handle larger heaps and reduces the cost of copying large, stationary objects. Enable it in `icp.yaml` under canister build args. Note: `--compacting-gc` applies only to the legacy classical persistence mode (`--legacy-persistence`); it is not used with the default enhanced orthogonal persistence.

#### Rust

**Exercise caution with `Vec<u8>` and `String` in state serialization.**

If you serialize/deserialize state that contains `Vec<u8>` or `String` values, Rust's memory layout requires copying each value during encoding and decoding. For large state, this increases the instruction cost significantly. Prefer `StableBTreeMap<Vec<u8>, ...>` (or a typed key) backed directly by stable memory over serializing heap collections on upgrade.

**Use `ic-stable-structures` for all persistent state.**

Put all important data in `StableBTreeMap`, `StableCell`, or `StableLog` from the start. This avoids the `pre_upgrade` serialization problem entirely. See [Implementing Storable for custom types](#store-data-durably) above for the correct pattern.

For reference on effective Rust canister patterns, see [Effective Rust Canisters](https://mmapped.blog/posts/01-effective-rust-canisters.html) and [How to audit an Internet Computer canister](https://www.joachim-breitner.de/blog/788-How_to_audit_an_Internet_Computer_canister).

### Implement state backup mechanisms

Even with stable memory, consider implementing explicit backup mechanisms for state that would be catastrophic to lose. This protects against:

- Accidental reinstall (which wipes stable memory)
- Bugs in upgrade hooks that corrupt the stable layout
- Application-level errors that require rollback

Common approaches include exporting a snapshot of canister state to a Blob that can be stored externally, or using canister [snapshots](../canister-management/snapshots.md) to checkpoint state before an upgrade.

### Transaction history storage

If your application needs to maintain a history of transactions or events, avoid storing unbounded logs in the same canister as your main application state. Options:

- **Dedicated logging canister.** A separate canister that accepts append-only log entries reduces load on the main canister and keeps the history size from affecting upgrade cost.
- **`StableLog` (Rust).** For canisters that can accommodate history growth, `StableLog` from `ic-stable-structures` provides an append-only log directly in stable memory.
- **External history services.** Services like [CAP](https://cap.ooo/) maintain transaction provenance records that integrate with explorers and wallets, which is useful for [digital asset standards](../../references/digital-asset-standards.md) compliance.

Be aware that inter-canister calls to a logging service add latency and cycle cost. Size the logging approach to the transaction volume you expect.

## Related

- [Orthogonal Persistence](../../concepts/orthogonal-persistence.md): conceptual explanation of heap vs. stable memory
- [Canister Lifecycle](../canister-management/lifecycle.md#what-happens-during-an-upgrade): upgrade hooks and canister lifecycle
- [Stable Structures (Rust)](../../languages/rust/stable-structures.md): deep dive into `ic-stable-structures`
- [Canister snapshots](../canister-management/snapshots.md): checkpoint canister state before upgrades
- [Motoko](../../languages/motoko/index.md): Motoko language overview and persistence model