Data Merkle Tree Generation

Sep 10, 2022 4:38 AM

This page describes the reasoning for applying Merkle Tree for Proof of Fair Randomization and method of which to verify

Sipher NFT Metadata - Merkle Tree Generationβ€Œ

Why is merkle tree of NFT data neededβ€Œ

As a generative art NFT project, several key concerns amongst consumers comes into mind:β€Œ

  1. The team wants the NFTs to have a flexible use-case while remaining entirely decentralized.
  2. Of the 10,000 SIPHER NFTs available, there would be a handful that possess a combination of rare traits, producing a β€œrare” NFT in comparison to the rest of the collection. The team is aiming to prevent the leak of such data before the metadata reveal as it serves as an influence on buying decisions.
  3. This ensures a level playing field as nobody has an advantage over the other.
  4. The team wants to save gas usage across the board.β€Œ

In order to address the first concern, the team has decided to host all metadata, including the generated photo files, attributes, and file names on a private server instead of IPFS, setting up a merkle tree over the metadata to verify the integrity of your NFT. For the other concerns, a sale in the form of a β€œloot box” drop is efficient as nobody has control over what is purchased. This results in lower gas prices as consumers would not go out of their way to β€œsnipe” a β€œrare” NFT. These mechanisms would be discussed in further detail under Sale Mechanism.β€Œ

Sale mechanismβ€Œ

To better understand how the merkle tree plays its role in the Sipher sale, let's run through how SIPHER's NFT public sale is conducted.β€Œ

Firstly, the team will deploy the sale's smart contract, where the team is required to submit the merkle root to the sale smart contract before any sales occurs (including internal team sales, whitelisted participants, and the public sale).β€Œ

Next, once the merkle root is set and the sale's star time kicks in, the team is allowed to buy up to 500 NFTs at 0 eth, while the other whitelisted addresses are allowed to buy 1 NFT per wallet at the price of 0.1 eth.β€Œ

At the end of public sale, the team is allowed to push through a transaction in order to roll randomizedStartIndex, a key factor to the random generation of the NFTs. Here, all the "loot boxes" are opened, technically mapped to the purchased NFTs within the SIPHER server in accordance to the following rule:

NFTSipherID = mod((tokenID + randomizedStartIndex βˆ’ 2), 10000 )+1


However, there are several technical concerns that arise as part of this sales flow, one that will be addressed below.


  1. Everyone will be allocated tokenIDs prior to its reveal. This is important for NFT holders as the ID is essential for verification. However, prior to the transaction to push for randomizedStartIndex, the mapping between NFT Sipher ID on Sipher Server and tokenID is left unestablished
  2. The team will buy 500 NFTs (eg. IDs 1 -500). While the ID numbers are fixed, the team is unaware of the traits behind each ID as the mapping is left unknown prior to the randomizedStartIndex transaction push.
  3. All NFT data is stored on the SIPHER servers with the exception of the tokenID that is publicly visible. This comes with a mutual trust between the team and the consumer as it facilitates future on-chain and off-chain blockchain operations for gameplay. Thus, trust is placed on the SIPHER servers to correctly return data in accordance to the ID of the purchased NFT.
  4. As it is costly to achieve true decentralization and flexibility for a NFT-based game, the team has chosen to strike a balance between both features:
    • Consumers should be able to verify the metadata of the purchased NFT.
    • The verification result should be accepted as such.
    • The team should be trusted to deliver accurate data.β€Œ

Here, the merkle tree will be utilized to facilitate the verification of the metadata. As the merkle root is required to pushed before sales begin, the team has to finalize the metadata prior to the start of the sale, where it can no longer be modified after.β€Œ

How do buyers verify the NFT data?β€Œ

The merkle tree comes in to facilitate such verifications. The merkle root is required to pushed before the sale can start, this means the team has to finalize the NFT data before the sale and they can't modify the data afterward.β€Œ

Below is a diagram explaining the Merkle Tree in principle and how to Verify

Merkle Tree Diagram and How to Verify True Randomization



What is this repoβ€Œ

This repository provides a tool for the team to calculate the NFT data's merkle root as well as for NFT holders to verify their NFT data.


Merkle tree constructionβ€Œ

Merkle tree in this repo is constructed in the same way with merkle tree js library with the following hash function:

const leaves = data.tokens.map((token) => {
  return hashOneToken(token);
const tree = new MerkleTree(leaves, keccak256, {sort: true});


Where hashOneToken is a function to hash 1 NFT data as the following pseudo-code:

id = Id in Sipher server of the NFT.
attributeNames = All of the NFT's attribute names sorted alphabetically.
attributeValues = All of the attribute values, in the corresponding order to attributeNames.
mainPhotoMD5 = MD5 of the main photo.
emotions = All of the NFT's emotions sorted alphabetically.
emotionMD5s = All of the MD5 of NFT's emotion photos, in corresponding order to emotions.
name = Name of the NFT.
origin = Origin of the NFT.
return hash = keccak256(abiEncode(
  id, attributeNames, attributeValues, mainPhotoMD5,
  emotions, emotionMD5, name, origin


How to run the tool


To Sipher teamβ€Œ

In order to calculate merkle root and proofs for all NFT data.β€Œ

1. set your current directory to the root directory of the repo. 2. npm install to install all of the dependencies. 3. node generateMerkleRoot.js -f [path_to_final_data] to calculate the merkle tree and write the output to merkle_data_sipher.json in the current directory. You can find merkleRoot as well as all of the proofs for each NFT. The merkleRoot is used to push to Sale contract.


To NFT holdersβ€Œ

In order to verify if your NFT data is correct manually.β€Œ

1. Query on NFT contract to get your id and the randomizedStartIndex. 2. Download all of the photos related to your NFT including main photo, emotion photos and get MD5 of them. 3. Get all of the necessary NFT's data from Sipher servers using your NFT's tokenUri(id) function. Verify your computed MD5s in step #2 to see if they match with what returned from Sipher servers. If they don't, the data is not correct, you should ask the team. 4. Get merkleRoot from Sale contract. 5. Get proofs from Sipher server. 6. Use this const verifyProof = require('./merkleDist/parseData').verifyProof; function to run merkle proof verification. If it returns True, the data is fine, otherwise it is not.


Note: The Sipher team will work on an automatic tool to do all of those steps just with 1 click in the future to save all of the effort. However, it is still important for every NFT holders to understand how they can do it without relying fully on Sipher team.


For step by step to verify your NFT with the code, visit here: