Skip to content

PSBT::from_tx() → extract_tx() drops TxOutWitness (output surjection/range proofs), breaks explicit output tx serialization #263

@gmikeska

Description

@gmikeska

Summary

When using Psbt::from_tx() followed by extract_tx() in rust-elements v0.25.x, the output witness data (TxOutWitness) is not properly serialized in the extracted transaction. This causes transactions with explicit (non-confidential) outputs to be rejected by Elements Core with "bad-txns-in-ne-out" (value in != value out).

Environment

  • rust-elements version: 0.25.2
  • Elements Core version: v23.3.1
  • Network: Liquid Testnet
  • Rust version: stable

Steps to Reproduce

1. Create a transaction with explicit outputs using PSBT

use elements::{
    confidential, AssetIssuance, LockTime, Script, Sequence, 
    Transaction, TxIn, TxInWitness, TxOut, TxOutWitness,
};
use elements::pset::PartiallySignedTransaction as Psbt;
use elements::encode::serialize_hex;

// Build an unsigned transaction with explicit (non-confidential) outputs
fn build_unsigned_tx() -> Transaction {
    let asset_id = elements::AssetId::from_str(
        "144c654344aa716d6f3abcc1ca90e5641e4e2a7f633bc09fe3baf64585819a49"
    ).unwrap();
    
    Transaction {
        version: 2,
        lock_time: LockTime::ZERO,
        input: vec![TxIn {
            previous_output: elements::OutPoint::new(/* txid */, 0),
            is_pegin: false,
            script_sig: Script::new(),
            sequence: Sequence::MAX,
            asset_issuance: AssetIssuance::null(),
            witness: TxInWitness::empty(),
        }],
        output: vec![
            // Destination output - explicit value
            TxOut {
                asset: confidential::Asset::Explicit(asset_id),
                value: confidential::Value::Explicit(50000),
                nonce: confidential::Nonce::Null,
                script_pubkey: /* destination script */,
                witness: TxOutWitness::empty(),  // Empty but should be serialized
            },
            // Fee output - explicit value
            TxOut::new_fee(500, asset_id),
        ],
    }
}

// Using PSBT to add witness and extract
fn finalize_with_psbt(witness_stack: Vec<Vec<u8>>) -> Transaction {
    let mut psbt = Psbt::from_tx(build_unsigned_tx());
    
    // Set the input witness (e.g., for Taproot/Simplicity)
    psbt.inputs_mut()[0].final_script_witness = Some(witness_stack);
    
    // Extract the final transaction
    psbt.extract_tx().unwrap()  // BUG: Output witnesses are dropped here
}

2. Serialize and broadcast

let tx = finalize_with_psbt(my_witness_stack);
let tx_hex = serialize_hex(&tx);

// Attempt to broadcast via Elements RPC
// Result: "bad-txns-in-ne-out, value in != value out"

3. Verify with decoderawtransaction

elements-cli decoderawtransaction "<tx_hex>"
# Earlier versions: "TX decode failed"
# After some fixes: Decodes but fails validation

Expected Behavior

The serialized transaction should include output witness data for each output. For explicit outputs, this should be:

00  ← surjection_proof length (0 = empty)
00  ← range_proof length (0 = empty)

For a 3-output transaction, this adds 6 bytes to the witness section:

00 00 00 00 00 00  ← 3 outputs × 2 empty proofs
04                 ← input witness stack count
...                ← input witness items

Actual Behavior

The extracted transaction is missing output witness bytes. The witness section starts with the input witness count immediately after locktime, causing malformed serialization:

...00000000  ← locktime
0004         ← only 2 bytes before input witness (should be 00 00 00 00 00 00 04)

This causes Elements Core to either:

  1. Fail to decode the transaction entirely, OR
  2. Misinterpret the byte boundaries, leading to "value in != value out"

Root Cause Analysis

When Psbt::from_tx() creates a PSBT from a Transaction, and then extract_tx() reconstructs the Transaction, the TxOutWitness fields on each TxOut are not being preserved or correctly transferred.

Looking at the PSBT output handling in rust-elements, the issue likely occurs in how output witness data flows through the PSBT round-trip.

Workaround

Build the Transaction directly without using PSBT:

fn finalize_directly(witness_stack: Vec<Vec<u8>>) -> Transaction {
    let input_witness = TxInWitness {
        amount_rangeproof: None,
        inflation_keys_rangeproof: None,
        script_witness: witness_stack,
        pegin_witness: vec![],
    };

    // Build Transaction directly - preserves TxOutWitness on outputs
    Transaction {
        version: 2,
        lock_time: LockTime::ZERO,
        input: vec![TxIn {
            previous_output: elements::OutPoint::new(/* txid */, 0),
            is_pegin: false,
            script_sig: Script::new(),
            sequence: Sequence::MAX,
            asset_issuance: AssetIssuance::null(),
            witness: input_witness,  // Set witness directly on TxIn
        }],
        output: outputs,  // TxOut structs with witness: TxOutWitness::empty()
    }
}

This ensures TxOutWitness::empty() on each output is properly serialized.

Test Case

#[test]
fn test_psbt_preserves_output_witness() {
    use elements::encode::{serialize, deserialize};
    
    // Build tx with explicit outputs
    let original_tx = build_unsigned_tx();
    
    // Round-trip through PSBT
    let psbt = Psbt::from_tx(original_tx.clone());
    let extracted_tx = psbt.extract_tx().unwrap();
    
    // Serialize both
    let original_bytes = serialize(&original_tx);
    let extracted_bytes = serialize(&extracted_tx);
    
    // The witness sections should match (or at least extracted should include output witnesses)
    // This test will fail with current implementation
    assert_eq!(original_bytes.len(), extracted_bytes.len());
}

Impact

  • Any Elements/Liquid transaction using PSBT with explicit (non-confidential) outputs will fail to broadcast
  • Particularly affects Taproot/Simplicity transactions where PSBT is commonly used to attach script witnesses
  • Workaround exists but requires avoiding PSBT entirely

Additional Context

This was discovered while building a Simplicity p2pkh spending implementation. The transaction:

  • 1 input: Taproot/Simplicity UTXO with 100,000 sats
  • 3 outputs: destination (50,000), change (49,500), fee (500) - all explicit
  • All values summed correctly but transaction was rejected

The issue was isolated by comparing our serialized transaction against one created by elements-cli createrawtransaction, which revealed the missing output witness bytes.

Related Files

  • Workaround implementation: musk/src/spend.rs - SpendBuilder::finalize_with_satisfied()
//! Transaction construction and spending utilities

use crate::client::Utxo;
use crate::error::SpendError;
use crate::program::{InstantiatedProgram, SatisfiedProgram};
use elements::hashes::Hash;
use elements::{
    confidential, AssetIssuance, LockTime, Script, Sequence, Transaction, TxIn, TxInWitness, TxOut,
    TxOutWitness,
};
use simplicityhl::simplicity::jet::elements::{ElementsEnv, ElementsUtxo};
use simplicityhl::WitnessValues;

/// Builder for constructing spending transactions
pub struct SpendBuilder {
    program: InstantiatedProgram,
    utxo: Utxo,
    outputs: Vec<TxOut>,
    lock_time: LockTime,
    sequence: Sequence,
    genesis_hash: elements::BlockHash,
}

impl SpendBuilder {
    /// Create a new spend builder for the given program and UTXO
    #[must_use]
    pub fn new(program: InstantiatedProgram, utxo: Utxo) -> Self {
        Self {
            program,
            utxo,
            outputs: Vec::new(),
            lock_time: LockTime::ZERO,
            sequence: Sequence::MAX,
            genesis_hash: elements::BlockHash::from_byte_array([0u8; 32]), // Default, should be set
        }
    }

    /// Set the genesis block hash (required for sighash computation)
    #[must_use]
    pub const fn genesis_hash(mut self, hash: elements::BlockHash) -> Self {
        self.genesis_hash = hash;
        self
    }

    /// Add an output to the transaction
    pub fn add_output(&mut self, output: TxOut) -> &mut Self {
        self.outputs.push(output);
        self
    }

    /// Add a simple output with explicit value
    pub fn add_output_simple(
        &mut self,
        script_pubkey: Script,
        amount: u64,
        asset: elements::AssetId,
    ) -> &mut Self {
        self.outputs.push(TxOut {
            value: confidential::Value::Explicit(amount),
            script_pubkey,
            asset: confidential::Asset::Explicit(asset),
            nonce: confidential::Nonce::Null,
            witness: TxOutWitness::empty(),
        });
        self
    }

    /// Add a fee output
    pub fn add_fee(&mut self, amount: u64, asset: elements::AssetId) -> &mut Self {
        self.outputs.push(TxOut::new_fee(amount, asset));
        self
    }

    /// Set the lock time
    #[must_use]
    pub const fn lock_time(mut self, lock_time: LockTime) -> Self {
        self.lock_time = lock_time;
        self
    }

    /// Set the sequence number
    #[must_use]
    pub const fn sequence(mut self, sequence: Sequence) -> Self {
        self.sequence = sequence;
        self
    }

    /// Compute the `sighash_all` for this transaction
    ///
    /// This is used to generate witness values that include signatures
    ///
    /// # Errors
    ///
    /// Returns an error if the control block cannot be found.
    pub fn sighash_all(&self) -> Result<[u8; 32], SpendError> {
        let tx = self.build_unsigned_tx();
        let utxo = ElementsUtxo {
            script_pubkey: self.utxo.script_pubkey.clone(),
            value: confidential::Value::Explicit(self.utxo.amount),
            asset: self.utxo.asset,
        };

        let (script, _version) = self.program.script_version();
        let control_block = self
            .program
            .taproot_info()
            .control_block(&(script, self.program.script_version().1))
            .ok_or_else(|| SpendError::BuildError("Control block not found".into()))?;

        let env = ElementsEnv::new(
            &tx,
            vec![utxo],
            0,
            self.program.cmr(),
            control_block,
            None,
            self.genesis_hash,
        );

        Ok(*env.c_tx_env().sighash_all().as_byte_array())
    }

    /// Build the unsigned transaction
    fn build_unsigned_tx(&self) -> Transaction {
        Transaction {
            version: 2,
            lock_time: self.lock_time,
            input: vec![TxIn {
                previous_output: elements::OutPoint::new(self.utxo.txid, self.utxo.vout),
                is_pegin: false,
                script_sig: Script::new(),
                sequence: self.sequence,
                asset_issuance: AssetIssuance::null(),
                witness: TxInWitness::empty(),
            }],
            output: self.outputs.clone(),
        }
    }

    /// Finalize the transaction with witness values
    ///
    /// # Errors
    ///
    /// Returns an error if the program cannot be satisfied or the transaction cannot be finalized.
    pub fn finalize(self, witness_values: WitnessValues) -> Result<Transaction, SpendError> {
        let satisfied = self.program.satisfy(witness_values)?;
        self.finalize_with_satisfied(&satisfied)
    }

    /// Finalize the transaction with a pre-satisfied program
    ///
    /// # Errors
    ///
    /// Returns an error if the control block cannot be found or transaction extraction fails.
    pub fn finalize_with_satisfied(
        self,
        satisfied: &SatisfiedProgram,
    ) -> Result<Transaction, SpendError> {
        let (script, version) = self.program.script_version();
        let control_block = satisfied
            .taproot_info()
            .control_block(&(script.clone(), version))
            .ok_or_else(|| SpendError::BuildError("Control block not found".into()))?;

        let (program_bytes, witness_bytes) = satisfied.encode();

        // Build the input witness stack for Simplicity/Taproot
        let input_witness = TxInWitness {
            amount_rangeproof: None,
            inflation_keys_rangeproof: None,
            script_witness: vec![
                witness_bytes,
                program_bytes,
                script.into_bytes(),
                control_block.serialize(),
            ],
            pegin_witness: vec![],
        };

        // Build the transaction directly (avoid PSBT which may drop output witnesses)
        Ok(Transaction {
            version: 2,
            lock_time: self.lock_time,
            input: vec![TxIn {
                previous_output: elements::OutPoint::new(self.utxo.txid, self.utxo.vout),
                is_pegin: false,
                script_sig: Script::new(),
                sequence: self.sequence,
                asset_issuance: AssetIssuance::null(),
                witness: input_witness,
            }],
            output: self.outputs,
        })
    }
}

/// Helper to create a simple spending transaction
///
/// # Errors
///
/// Returns an error if the asset is not explicit or the transaction cannot be built.
pub fn simple_spend(
    program: InstantiatedProgram,
    utxo: Utxo,
    destination: Script,
    amount: u64,
    fee: u64,
    genesis_hash: elements::BlockHash,
    witness_values: WitnessValues,
) -> Result<Transaction, SpendError> {
    let confidential::Asset::Explicit(asset) = utxo.asset else {
        return Err(SpendError::InvalidUtxo("Non-explicit asset".into()));
    };

    let mut builder = SpendBuilder::new(program, utxo).genesis_hash(genesis_hash);
    builder.add_output_simple(destination, amount, asset);
    builder.add_fee(fee, asset);
    builder.finalize(witness_values)
}

Possibly related issue

#262 "Unable to decode serialized TxOut"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions