Generating 10 Million Unique IDs: Which Is Faster, UUID, ULID, or NanoID?

Today i was reading about ULID and became curious about the performance differences between generating IDs using UUID, ULID, and NanoID. I had already written about NanoID here: https://medium.com/@prabeshthapa/rethinking-unique-identifiers-exploring-nanoid-as-an-alternative-to-uuids-a5fb4cef20f6

I knew it was fast but wanted to see how would it compare compared to other UUID and NanoID. While they share the common goal of uniqueness, they differ significantly in structure, purpose, and use cases.

UUID (Universally Unique Identifier)

Definition: A UUID is a 128-bit identifier designed to provide a unique value across distributed systems without requiring a central coordinating authority.

Structure:

• Composed of 32 hexadecimal characters, typically represented in a standard format: 8-4-4-4-12 (e.g., 123e4567-e89b-12d3-a456-426614174000).
• Contains segments for different purposes, including time-based and random values.

Versions:

• UUIDv1: Includes a timestamp and MAC address, potentially exposing the host’s identity and time.
• UUIDv4: Randomly generated, offering high entropy and privacy but no inherent ordering.
• UUIDv5: Uses a namespace and a name to create a consistent identifier across different systems.

Advantages:

• Uniqueness: High probability of uniqueness across time and space.
• Widely Supported: Commonly used in databases, APIs, and distributed systems.
• Standardized: RFC 4122 defines the format and generation rules, ensuring interoperability.

Disadvantages:

• Not Ordered: UUIDs are not lexicographically sortable by default, which can be a downside for indexing and querying in databases.
• Complexity: UUIDs are relatively large and complex, which might not be ideal for some use cases where simplicity and readability are preferred.

ULID (Universally Unique Lexicographically Sortable Identifier)

Definition: A ULID is a 128-bit identifier that combines uniqueness with lexicographical sorting, making it suitable for scenarios requiring both unique identification and natural order.

Structure:

• Composed of 26 alphanumeric characters in Base32 format (e.g., 01ARZ3NDEKTSV4RRFFQ69G5FAV).
• Includes a timestamp and a random component.

Advantages:

• Lexicographical Order: ULIDs are sortable by creation time, facilitating efficient queries and ordered data storage.
• Compact: Shorter and more human-readable than UUIDs.
• Base32 Encoding: Case-insensitive and avoids confusion with commonly mistaken characters.

Disadvantages:

• Not Standardized: There is no formal standard like RFC for ULIDs, leading to potential inconsistencies in implementations.
• Limited Language Support: Compared to UUIDs, fewer libraries and languages have built-in support for ULIDs.

NanoID

Definition: NanoID is a modern, URL-friendly, unique string identifier designed to be efficient and compact. It aims to generate unique IDs that are short and easy to use in various applications, particularly in web and database contexts.

Structure:

• NanoID generates strings with a customizable length, typically 21 characters, using a default alphabet of 64 URL-safe characters (A-Za-z0-9_-).
• The compactness of NanoID makes it ideal for environments where URL-friendly and concise identifiers are required.

Customization:

• Length: The length of the NanoID can be customized to balance between uniqueness and storage efficiency. The default length of 21 characters provides sufficient uniqueness for most use cases.
• Alphabet: The character set used to generate the ID can be customized, allowing the creation of IDs that fit specific constraints or requirements (e.g., URL-safe, no ambiguous characters).

Advantages:

• Compact and Efficient: NanoID’s default length of 21 characters is shorter than UUIDs, making it more efficient in terms of storage and transmission.
• URL-Friendly: The default character set is URL-safe, which makes NanoIDs ideal for web applications where IDs might be included in URLs.
• Customizable: Both the length and the character set of NanoID can be adjusted to meet specific requirements, offering flexibility for different use cases.

Disadvantages:

• Not Standardized: Unlike UUIDs, NanoID does not have an official standard, which might lead to inconsistencies in implementation across different platforms.
• Entropy Management: Customizing the length and alphabet requires careful consideration to ensure that the generated IDs maintain a high probability of uniqueness.
• Less Widely Supported: Compared to UUIDs, NanoID might have less out-of-the-box support in existing libraries and systems, potentially requiring custom implementation.

Key Differences

UUID

• 128 bits
• Hexadecimal format
• Not inherently sortable
• High entropy
• Not URL Friendly
• Optional timestamp
• Widely used

ULID

• 128 bits ( 26 chars )
• Base32 format
• Lexicographically sortable
• Not standardized ( early )
• Partially timestamp based
• Includes a timestamp
• Efficient ordering and sorting

NanoID

• Customizable ( 21 Chars )
• Alphanumeric
• Adjustable length and alphabet
• URL friendly
• Not standarised
• Web App and custom ID

Use Cases

UUID:

• Distributed Systems: Ensures unique identifiers without a central authority.
• Databases: Suitable for keys where the order is not a concern.
• APIs: Widely accepted as a standard for unique resource identifiers.

ULID:

• Ordered Data Storage: Ideal for databases where the order of entries matters.
• Event Logging: Helps in maintaining a chronological order of events.
• URL Shorteners: Useful where a compact and readable identifier is needed.

NanoID:

• Web Applications: Perfect for generating compact, URL-safe identifiers for web resources.
• URLs: Used in short links, user identifiers, and other scenarios where a short, unique string is beneficial.
• Custom ID Requirements: Ideal for applications needing unique IDs with specific length or character set constraints.

Let’s generate some unique IDs

Here are the sample code to generate UUID, ULID and NanoID using Go. This is generic implementation from well-known packages. These codes are executed on a Apple M1 Pro with 32 GB of Memory.

UUID

package main

import (
 "fmt"
 "time"

 "github.com/google/uuid"
)

func main() {
 const runs = 1000
 const idsPerRun = 10

 var totalDuration time.Duration

 for j := 0; j < runs; j++ {
  start := time.Now()

  for i := 0; i < idsPerRun; i++ {
   id := uuid.New()
   _ = id 
  }

  end := time.Now()
  duration := end.Sub(start)
  totalDuration += duration
  fmt.Printf("Run %d: Execution Time: %v\n", j+1, duration)
 }

 averageDuration := totalDuration / time.Duration(runs)
 fmt.Printf("Average Execution Time over %d runs: %v\n", runs, averageDuration)
}

ULID

package main

import (
 "fmt"
 "math/rand"
 "time"

 "github.com/oklog/ulid/v2"
)

func main() {
 const runs = 1000
 const idsPerRun = 10

 var totalDuration time.Duration

 source := rand.NewSource(time.Now().UnixNano())
 r := rand.New(source)

 for j := 0; j < runs; j++ {
  start := time.Now()

  for i := 0; i < idsPerRun; i++ {
   id := ulid.MustNew(ulid.Timestamp(time.Now()), r)
   _ = id
  }

  end := time.Now()
  duration := end.Sub(start)
  totalDuration += duration
  fmt.Printf("Run %d: Execution Time: %v\n", j+1, duration)
 }

 averageDuration := totalDuration / time.Duration(runs)
 fmt.Printf("Average Execution Time over %d runs: %v\n", runs, averageDuration)
}

NanoID

package main

import (
 "fmt"
 "time"

 nanoid "github.com/matoous/go-nanoid/v2"
)

func main() {
 const runs = 1000
 const idsPerRun = 10

 var totalDuration time.Duration

 for j := 0; j < runs; j++ {
  start := time.Now()

  for i := 0; i < idsPerRun; i++ {
   id, err := nanoid.New()
   if err != nil {
    fmt.Println("Error generating NanoID:", err)
    return
   }
   _ = id
  }

  end := time.Now()
  duration := end.Sub(start)
  totalDuration += duration
  fmt.Printf("Run %d: Execution Time: %v\n", j+1, duration)
 }

 averageDuration := totalDuration / time.Duration(runs)
 fmt.Printf("Average Execution Time over %d runs: %v\n", runs, averageDuration)
}

Execution time analysis

The performance of generating unique IDs was compared across three different types: UUID, ULID, and NanoID, with the following findings:

Generating 10 unique IDs

Total Execution Time: 52.208µs // ULID
Total Execution Time: 60.209µs // UUID
Total Execution Time: 27.5µs // NanoID

Observation: NanoID was the fastest, taking significantly less time than both ULID and UUID.

Generating 1000 unique IDs

Total Execution Time: 888.916µs // ULID
Total Execution Time: 1.447833ms // UUID
Total Execution Time: 1.892417ms // NanoID

Observation: ULID outperformed both UUID and NanoID, with UUID being slightly faster than NanoID.

Generating 1000000 ( 1 Million ) unique IDs

Total Execution Time: 1.328692208s // ULID
Total Execution Time: 2.052411625s // UUID
Total Execution Time: 657.343333ms // NanoID

Observation: NanoID was the most efficient, completing the task in less than half the time of ULID and significantly faster than UUID.

Generating 10000000 ( 10 Million ) unique IDs

Total Execution Time: 30.053901625s // ULID
Total Execution Time: 42.146729583s // UUID
Total Execution Time: 6.280103291s // NanoID

Observation: NanoID demonstrated superior performance, being almost five times faster than ULID and nearly seven times faster than UUID.

Finding

As we saw that ULID outperformed while generating 1000 IDs due to its efficient timestamp-based generation and lower initial overhead. However, as the number of IDs increases (e.g., 1 million), the cumulative complexity of maintaining unique timestamps and generating random components makes ULID’s advantages less pronounced. In contrast, UUID and NanoID, while initially slower, scale more consistently due to their reliance on random number generation and the specifics of their implementation. This results in ULID losing its edge in large-scale ID generation tasks.

While there is not a significant difference in performance when generating a small number of IDs, the real advantages of NanoID and ULID become apparent as the data volume increases. NanoID, in particular, shows substantial performance gains, making it the most efficient option for large-scale ID generation. ULID also offers better performance compared to UUID, which was the slowest across all tested scales.

Conclusion

Even if UUIDs are a standard choice for generating unique identifiers, they may not always be the best option, especially when high performance and efficiency are required. NanoID and ULID offer significant performance advantages, particularly at large scales.

Choosing between UUID, ULID or NanoID depends on the specific requirements of your project. If you need a universally accepted, robust identifier without a requirement for ordering, a UUID is an excellent choice. However, if your use case benefits from a time-ordered, human-readable identifier, a ULID might be more appropriate. If you want performance NanoID might be best for you. Each has its strengths and should be selected based on the needs of your system.

P.S: If you want to get these article delivered to your mailbox, then please subscribe to my SRE/DevOps Newsletter: ( https://reliabilityengineering.substack.com ).