Skip to content

Latest commit

 

History

History
55 lines (33 loc) · 2.09 KB

File metadata and controls

55 lines (33 loc) · 2.09 KB

Token embeddings

Input embedding layers, paired with the corresponding block type.

Registry: praxis.EMBEDDING_REGISTRY (15 entries)

byte

Value: functools.partial(<function praxis.embeddings._compose>, [('tok', {})])

byte_hash

Value: functools.partial(<function praxis.embeddings._compose>, [('tok', {}), ('hash', {'group_sizes': [3, 4, 5], 'functions': 1})])

byte_multihash

Value: functools.partial(<function praxis.embeddings._compose>, [('tok', {}), ('hash', {'group_sizes': [3, 4, 5], 'functions': 4})])

conv, gru, min, nano, recurrent, ssm, transformer, wavelet - ProjectedEmbedding

An embeddings module with optional projection layer and dropout. If embed_size differs from hidden_size, a linear projection layer is added to map the embeddings to the required hidden dimension.

Source: praxis/embeddings/projected.py:11

hash - HashEmbedding

N-gram hash embedding: sums table lookups over byte windows of several sizes and hash functions, computing vectors from byte n-grams rather than retrieving a per-token row. Has no single tie-able table by design.

Collision resistance comes from functions, not from hash_vocab. A single hash maps every distinct n-gram onto exactly one row, so two n-grams sharing a bucket become the same vector and are unrecoverable downstream; the only defence is a wider table, and byte n-grams outnumber ...

Source: praxis/embeddings/hash.py:41

mru, positional - PositionalEmbedding

Praxis embeddings with learned positional encodings (GPT2-style). Uses Sequential organization of layers.

Source: praxis/embeddings/positional.py:11

tok - ByteEmbedding

Per-byte token table for byte-latent encoders; the weight-tying target.

Sizes itself from the encoder's declared input layout (input_dim / input_vocab_size) when present, else from the global config.

Source: praxis/embeddings/byte.py:4