codec

package module
v1.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 18, 2025 License: MIT Imports: 11 Imported by: 0

README

Codec: High-Performance Binary Serialization

Go Report Card

中文 | English

codec is a binary serialization library designed for high-performance network protocols and file format parsing in Go.

It addresses the performance bottlenecks (heavy reflection) and verbosity associated with the standard library's encoding/binary. By leveraging Generics and Buffer Pooling, codec provides Zero-Allocation read/write capabilities and a robust Streaming interface.

It is specifically tailored for building TCP protocol stacks, file parsers, or any scenario requiring ultra-low latency binary data processing.

Core Features

  • High-Performance Fixed-Size Encoding:
    • Uses Fixed[T] generics to automatically cache struct sizes, avoiding repetitive reflection overhead (binary.Size).
    • Eliminates the boilerplate code of manual binary.Write calls.
  • Enhanced Stream I/O:
    • Error Latching: The Reader and Writer capture the first error encountered, allowing for fluent, chained calls without checking if err != nil at every step.
    • Smart Buffering: Intelligently wraps bufio to prevent performance degradation caused by Double Buffering.
  • Zero-Copy Optimization:
    • Provides BytesReader and BytesWriter to operate directly on memory slices.
    • Fully implements io.ReaderFrom and io.WriterTo to utilize underlying optimizations (like sendfile or memory copy).
  • Complex Structure Support:
    • List[T]: Native support for encoding/decoding slices of structs with built-in Memory Alignment (e.g., 4-byte/8-byte padding).
    • ChainedReader: A specialized reader for handling streams with Trailers (data following a main payload).
  • Safety & Robustness:
    • CheckTrailingNotZeros: Validates that no garbage data remains after parsing.
    • MaxPadding: Protects against malicious excessive padding attacks.

Installation

go get github.com/oy3o/codec

Quick Start

1. Fixed-Size Structs

For structs with fixed-length fields, Fixed[T] delivers maximum performance with minimal code.

package main

import (
    "fmt"
    "github.com/oy3o/codec"
)

// Define your protocol header. Must contain fixed-size fields only.
type Header struct {
    Magic   uint32
    Version uint16
    OpCode  uint16
    BodyLen uint64
}

func main() {
    // 1. Create the codec wrapper
    h := codec.Fixed[Header]{
        Payload: Header{
            Magic:   0xCAFEBABE,
            Version: 1,
            OpCode:  101,
            BodyLen: 500,
        },
    }

    // 2. Marshal (Serialize)
    // The first call calculates and caches the size; subsequent calls have zero reflection overhead.
    data, _ := h.MarshalBinary()
    fmt.Printf("Size: %d bytes, Hex: %x\n", h.Size(), data)

    // 3. Unmarshal (Deserialize)
    var h2 codec.Fixed[Header]
    _ = h2.UnmarshalBinary(data)
    fmt.Printf("Decoded: %+v\n", h2.Payload)
}
2. Streaming & Lists with Alignment

Handle complex network streams with automatic padding alignment.

func handleConnection(conn net.Conn) {
    // 1. Create a buffered Writer
    w, _ := codec.NewWriter(conn)
    
    // 2. Fluent Writing (Error Latching)
    // You don't need to check errors after every line.
    w.WriteUint32(0xDEADBEEF) // Magic
    w.WriteUint8(1)           // Version
    
    // 3. Write a List (Align to 4 bytes)
    items := []codec.Codec{
        &codec.Fixed[Item]{Payload: Item{ID: 1}},
        &codec.Fixed[Item]{Payload: Item{ID: 2}},
    }
    list := codec.NewList4(items) // Auto-handles padding between items
    w.WriteFrom(list)

    // 4. Commit and Check Error
    // Flush the buffer and check the first error that occurred (if any).
    if _, err := w.Result(); err != nil {
        log.Println("Write failed:", err)
        return
    }
}
3. Advanced Stream Processing (ChainedReader)

Handle protocols designed as "Payload + Trailer".

func readStream(r io.Reader) {
    // Assume the first 100 bytes are the main payload
    payloadSize := 100
    
    // Define behavior after payload is consumed (e.g., verify signature or read trailer)
    onPayloadDone := func(trailerReader io.Reader) error {
        // Read the remaining trailer data
        trailerData, _ := io.ReadAll(trailerReader)
        fmt.Println("Trailer read:", len(trailerData))
        return nil
    }

    // Create a ChainedReader
    cr := codec.ChainReader(r, int64(payloadSize), onPayloadDone)

    // Read the main body
    body := make([]byte, payloadSize)
    io.ReadFull(cr, body) 
    
    // When the body is fully read, `onPayloadDone` triggers automatically.
}

Architecture

codec balances flexibility and performance using a set of granular interfaces.

classDiagram
    class Sizer {
        +Size() int
    }
    class Marshaler {
        +MarshalBinary() ([]byte, error)
        +WriteTo(w io.Writer) (int64, error)
        +MarshalTo(buf []byte) (int, error)
    }
    class Unmarshaler {
        +UnmarshalBinary(data []byte) error
        +ReadFrom(r io.Reader) (int64, error)
    }
    class Codec {
        <<interface>>
    }
    
    Sizer <|-- Codec
    Marshaler <|-- Codec
    Unmarshaler <|-- Codec
    
    class Fixed_T {
        +Payload T
        -sizeCache
    }
    Codec <|.. Fixed_T : Implements

Components

Fixed[Payload]

The core of the library. It uses Go generics to wrap fixed-size structs (structs without slice, map, or string fields).

  • Size Cache: Uses xsync.Map to safely cache type sizes concurrently, making Size() calls extremely fast.
  • Generic Implementation: Automatically implements the Codec interface for any compliant struct.
Writer / Reader

Advanced wrappers around standard io interfaces.

  • Typed Operations: Methods like WriteUint32, ReadInt64, WriteBool with configurable ByteOrder (BigEndian/LittleEndian).
  • Error Latching: Maintains an internal err state. If an error occurs (e.g., io.ErrShortWrite), subsequent operations become no-ops until Result() or Flush() is called.
List[T]

Handles slices of Codec items.

  • Alignment: Factory methods like NewList4 and NewList8 automatically insert Zero Padding bytes between elements to satisfy protocol alignment requirements.
CheckTrailingNotZeros

A security utility. Automatically called at the end of UnmarshalBinary. It checks if any non-zero bytes remain in the reader, which is critical for detecting packet truncation, parsing bugs, or protocol smuggling attacks.

Interfaces

To integrate custom types into the codec ecosystem, simply implement the Codec interface:

type Codec interface {
    Sizer
    Marshaler
    Unmarshaler
}

type Sizer interface {
    Size() int
}

type Marshaler interface {
    encoding.BinaryMarshaler
    io.WriterTo
    MarshalTo(buf []byte) (int, error) // Zero-allocation write
}

type Unmarshaler interface {
    encoding.BinaryUnmarshaler
    io.ReaderFrom
}

Documentation

Index

Constants

View Source 
const BUFFER_SIZE = 4096
View Source 
const CHUNK_SIZE = 32 * 1024
View Source 
const MAX_PADDING = 1024 // 1KB

MAX_PADDING defines the maximum number of trailing bytes to check. This prevents an Out-Of-Memory error if a parsing bug leaves a large amount of data in the reader. Anything larger is considered a protocol error.

Variables

View Source 
var (
	// ErrNilIO indicates that NewReader/NewWriter was called with an nil interface
	ErrNilIO = errors.New("codec: NewReader/NewWriter called with a nil io.Reader/io.Writer")

	// ErrSizeTooSmall indicates a size conflict with bufio
	ErrSizeTooSmall = errors.New("codec: NewReaderSize with a size smaller than 16 conflict with bufio")

	// ErrAlreadyBuffered indicates that NewReader/NewWriter was called with an already-buffered
	// reader/writer, which would lead to unpredictable behavior and performance issues.
	ErrAlreadyBuffered = errors.New("codec: reader or writer is already buffered")

	// ErrWriteToNil indicates a WriteTo operation was attempted on a nil io.Writer.
	ErrWriteToNil = errors.New("codec: WriteTo called with a nil io.Writer")

	// ErrReadToNil indicates a ReadTo operation was attempted on a nil io.ReaderFrom.
	ErrReadToNil = errors.New("codec: ReadTo called with a nil io.ReaderFrom")

	// ErrInvalidSeek indicates a seek was attempted to invalid position.
	ErrInvalidSeek = errors.New("codec: seek to a invalid position")

	// ErrUnsupportedNegativeSeek indicates a backward seek was attempted on a forward-only seeker.
	ErrUnsupportedNegativeSeek = errors.New("codec: unsupported negative offset for forward-only seeker")

	// ErrInvalidWhence indicates that an invalid 'whence' parameter was provided to a Seek operation.
	ErrInvalidWhence = errors.New("codec: unsupported whence for forward-only seeker")

	// ErrInvalidWrite indicates that an io.Writer returned an invalid (negative) count from Write.
	ErrInvalidWrite = errors.New("codec: writer returned invalid count from Write")

	// ErrInvalidRead indicates that an io.Reader returned an invalid (negative or outbound) count from Read.
	ErrInvalidRead = errors.New("codec: reader returned invalid count from Read")

	// ErrDiscardNegative indicates a Discard operation was attempted with a negative byte count.
	ErrDiscardNegative = errors.New("codec: cannot discard negative number of bytes")

	// ErrTrailingData is returned by UnmarshalBinaryGeneric when non-zero bytes are found
	// after the expected end of the data structure, indicating a potential parsing error or malformed data.
	ErrTrailingData = errors.New("codec: non-zero trailing data found after decoding")

	// ErrTruncatedData indicates that a read operation could not complete because the
	// underlying data source (e.g., buffer, stream) ended before all expected bytes were read.
	ErrTruncatedData = errors.New("codec: truncated data")
)
View Source 
var (
	BE = binary.BigEndian
	LE = binary.LittleEndian
	// Order is default binary order
	Order = BE
)
View Source 
var Zero io.Reader = zero{}

Zero is an io.Reader that reads an infinite stream of zero bytes.

Functions

func ChainReader 

func ChainReader(reader io.Reader, n int64, callback ChainedReaderCallback) reader

ChainReader creates a new ChainedReader which also satisfies io.ReadCloser. If the provided reader also implements io.Seeker, the returned value will support seeking as well.

reader: The underlying io.Reader. n: The number of bytes in the main data stream. callback: The function to execute after n bytes have been read.

func CheckBufferNotZeros added in v1.0.2 

func CheckBufferNotZeros(trailingData []byte) error

func CheckTrailingNotZeros 

func CheckTrailingNotZeros(r io.Reader) error

CheckTrailingNotZeros verifies that any remaining bytes in a reader are all zero. This is critical for parsers to ensure the entire expected payload was consumed and no garbage data follows, which could indicate a bug or a malicious payload.

func Discard 

func Discard(r io.Reader, n int64) (int64, error)

func ForwardSeekCloser 

func ForwardSeekCloser(r io.ReadCloser) io.ReadSeekCloser

ForwardSeekCloser wraps an io.ReadCloser to make it a forward-only io.ReadSeekCloser. If the provided reader already implements io.ReadSeekCloser, it is returned directly.

func ForwardSeeker 

func ForwardSeeker(r io.Reader) io.ReadSeeker

ForwardSeeker wraps an io.Reader to make it a forward-only io.ReadSeeker. If the provided reader already implements io.ReadSeeker, it is returned directly.

func LimitReader 

func LimitReader(r io.Reader, n int64) reader

func MarshalBinaryGeneric 

func MarshalBinaryGeneric[T interface {
	Size() int
	io.WriterTo
}](v T) ([]byte, error)

MarshalBinaryGeneric provides a generic `encoding.BinaryMarshaler` implementation.

func MarshalToGeneric 

func MarshalToGeneric[T interface {
	Size() int
	io.WriterTo
}](v T, p []byte) (int, error)

MarshalToGeneric provides a fallback implementation for the MarshalTo method.

func NewList 

func NewList[T Codec](items []T, options *listOptions) *list[T]

NewList creates a new List codec with the given items and options.

func Ptr 

func Ptr[T any](v T) *T

func ReadFromGeneric 

func ReadFromGeneric[T encoding.BinaryUnmarshaler](v T, r io.Reader) (int64, error)

ReadFromGeneric provides a generic, non-streaming `io.ReaderFrom` implementation. WARNING: This is NOT a streaming implementation. It reads the entire `io.Reader` into a memory buffer before unmarshalling. It is unsuitable for very large inputs.

func ReadUTF16StringUntilNull 

func ReadUTF16StringUntilNull(r io.Reader) (string, int64, error)

ReadUTF16StringUntilNull reads a null-terminated UTF-16 string from an io.Reader. It detects endianness from the BOM. If no BOM is present, it defaults to Little Endian. It correctly handles surrogate pairs for characters outside the Basic Multilingual Plane.

It returns the decoded UTF-8 string, the number of bytes read, and any error.

func ReadUntilNullOrEOF 

func ReadUntilNullOrEOF(r io.Reader) ([]byte, int64, error)

ReadStringUntilNullOrEOF reads bytes from an io.Reader until a null character (0) or an io.EOF is encountered.

To ensure efficient reading, this function first checks if the provided reader implements the io.ByteReader interface. If so, it uses the efficient ReadByte() method.

func ReadVariableField 

func ReadVariableField(r io.Reader, currentReadOffset, targetFieldOffset, targetFieldLength int64) ([]byte, int64, error)

ReadVariableField is a helper function to read a variable-length field from an stream. This function is a convenience wrapper around ReadVariableFieldStream, using an in-memory buffer as the destination. It is suitable for smaller payloads. For large payloads, ReadVariableFieldStream should be used directly to avoid memory allocation.

Parameters and return values are the same as the original function.

func ReadVariableFieldStream 

func ReadVariableFieldStream(dst io.Writer, r io.Reader, currentReadOffset, targetFieldOffset, targetFieldLength int64) (int64, error)

ReadVariableFieldStream is a helper function to read a variable-length field from an stream and stream its content directly to a destination writer. It avoids allocating memory for the field's data, making it suitable for large payloads like file transfers.

Parameters:

  • dst: The io.Writer where the field's data will be written. If nil, data is discarded.
  • r: The io.Reader to read from (the packet stream).
  • currentReadOffset: The total number of bytes read from the start of the packet so far.
  • targetFieldOffset: The offset of the desired field from the start of the packet.
  • targetFieldLength: The length of the desired field.

Returns:

  • int64: The total number of bytes consumed from the source reader 'r' (including padding).
  • error: An error if reading fails or the packet is malformed.

func Roundup 

func Roundup[T constraints.Integer](n, align T) T

Roundup rounds n up to the nearest multiple of align.

func UnmarshalBinaryGeneric 

func UnmarshalBinaryGeneric[T interface {
	io.ReaderFrom
	Size() int
}](v T, data []byte) error

UnmarshalBinaryGeneric provides a generic `UnmarshalBinary` for types implementing `io.ReaderFrom`. It adapts a stream-based `ReadFrom` to the slice-based `UnmarshalBinary` interface and adds a crucial check for unexpected trailing data.

func WriteToGeneric 

func WriteToGeneric[T encoding.BinaryMarshaler](v T, w io.Writer) (int64, error)

WriteToGeneric provides a generic `io.WriterTo` implementation. It adapts a type that can marshal to a byte slice to the streaming io.Writer interface.

Types

type BytesReader 

type BytesReader struct {
	B []byte // destination slice
	N int    // current read position
}

BytesReader is an io.Reader that read from a pre-allocated byte slice.

func NewBytesReader 

func NewBytesReader(b []byte) *BytesReader

NewBytesReader creates a new BytesReader.

func (*BytesReader) Available 

func (r *BytesReader) Available() int

Available returns the number of bytes available for reading.

func (*BytesReader) Close 

func (r *BytesReader) Close() error

Close closes the underlying reader if it implements io.Closer.

func (*BytesReader) Len 

func (w *BytesReader) Len() int

Len returns the number of bytes read.

func (*BytesReader) Next added in v1.0.3 

func (r *BytesReader) Next(n int) []byte

Next returns the next n bytes from the reader.

func (*BytesReader) Read 

func (r *BytesReader) Read(p []byte) (int, error)

Read implements the io.Reader interface.

func (*BytesReader) ReadByte 

func (r *BytesReader) ReadByte() (byte, error)

ReadByte implements the io.ByteReader interface.

func (*BytesReader) Reset 

func (w *BytesReader) Reset()

Reset allows the underlying byte slice to be reused.

func (*BytesReader) Seek 

func (r *BytesReader) Seek(offset int64, whence int) (int64, error)

Seek implements the io.Seeker interface.

func (*BytesReader) Size 

func (r *BytesReader) Size() int

Size returns the size of the underlying byte slice.

func (*BytesReader) WriteTo 

func (r *BytesReader) WriteTo(w io.Writer) (int64, error)

WriteTo implements the io.WriterTo interface for efficiency.

type BytesWriter 

type BytesWriter struct {
	B []byte // destination slice
	N int    // current write position
}

BytesWriter is an io.Writer that writes to a pre-allocated byte slice. It will not grow the slice's capacity. If a write exceeds the available space, it writes as much as it can and returns io.ErrShortWrite.

func NewBytesWriter 

func NewBytesWriter(p []byte) *BytesWriter

NewBytesWriter creates a new BytesWriter.

func (*BytesWriter) Available 

func (w *BytesWriter) Available() int

Available returns the number of bytes available for writing.

func (*BytesWriter) Bytes 

func (w *BytesWriter) Bytes() []byte

Bytes returns a slice view of the written data.

func (*BytesWriter) Close 

func (w *BytesWriter) Close() error

Close closes the underlying writer if it implements io.Closer.

func (*BytesWriter) Flush 

func (w *BytesWriter) Flush() error

Flush do nothing

func (*BytesWriter) Len 

func (w *BytesWriter) Len() int

Len returns the number of bytes written.

func (*BytesWriter) ReadFrom 

func (w *BytesWriter) ReadFrom(r io.Reader) (int64, error)

ReadFrom implements the io.ReaderFrom interface and reads data from r until EOF or an error occurs.

func (*BytesWriter) Reset 

func (w *BytesWriter) Reset()

Reset allows the underlying byte slice to be reused.

func (*BytesWriter) Size 

func (w *BytesWriter) Size() int

Size returns the capacity of the underlying byte slice.

func (*BytesWriter) Write 

func (w *BytesWriter) Write(p []byte) (int, error)

Write implements the io.Writer interface.

func (*BytesWriter) WriteByte 

func (w *BytesWriter) WriteByte(c byte) error

WriteByte implements the io.ByteWriter interface for efficiency.

func (*BytesWriter) WriteString 

func (w *BytesWriter) WriteString(s string) (int, error)

WriteString implements the io.StringWriter interface for efficiency.

func (*BytesWriter) WriteZeros 

func (w *BytesWriter) WriteZeros(n int) (int, error)

type ChainedReadSeeker 

type ChainedReadSeeker struct {
	*ChainedReader
	S io.Seeker
}

ChainedReadSeeker embeds a ChainedReader to add seeking capability. It is returned by ChainReader when the underlying reader implements io.Seeker.

func (*ChainedReadSeeker) Seek 

func (r *ChainedReadSeeker) Seek(offset int64, whence int) (int64, error)

Seek implements the io.Seeker interface. It adjusts the position within the stream and resets the state of the ChainedReader accordingly.

type ChainedReader 

type ChainedReader struct {
	U io.Reader             // U is the underlying raw stream (e.g., a net.Conn or os.File).
	R *io.LimitedReader     // R limits reading to the length of the main data stream.
	C ChainedReaderCallback // C is the callback to execute after the main stream is read.
	E bool                  // E (executed) is a flag to ensure the callback runs only once.
	N int64                 // N stores the original length of the main stream for seeking purposes.
}

ChainedReader is a reader that wraps an underlying stream. It reads a predefined number of bytes (the main stream) and then executes a callback action on the remainder of the stream. This is useful for handling data formats where a payload is followed by a trailer or metadata.

func (*ChainedReader) Close 

func (r *ChainedReader) Close() error

Close closes the underlying reader if it implements io.Closer.

func (*ChainedReader) Read 

func (r *ChainedReader) Read(p []byte) (n int, err error)

Read implements the io.Reader interface. It reads from the main stream. When the main stream is exhausted, it executes the chained callback before returning io.EOF.

func (*ChainedReader) WriteTo 

func (r *ChainedReader) WriteTo(w io.Writer) (n int64, err error)

WriteTo implements io.WriterTo for efficient copying, like in io.Copy.

type ChainedReaderCallback 

type ChainedReaderCallback func(trailerReader io.Reader) error

ChainedReaderCallback is the function type for an action to be executed after the main data stream has been fully consumed. It receives the underlying reader to continue reading from the stream (e.g., to process a trailer).

type Codec 

type Codec interface {
	Sizer
	Marshaler
	Unmarshaler
}

Codec aggregates all binary serialization and deserialization interfaces. A type implementing Codec is a complete, self-sizing binary encoder/decoder.

type Fixed 

type Fixed[Payload any] struct {
	Payload Payload
}

Fixed provides a generic `Codec` implementation for any struct `Body` composed of fixed-size fields, eliminating boilerplate for simple data structures.

Constraint: The `Body` type MUST NOT contain variable-size fields like slices, maps, or strings, as this will cause `binary.Size` to fail.

func (*Fixed[Payload]) MarshalBinary 

func (c *Fixed[Payload]) MarshalBinary() ([]byte, error)

MarshalBinary implements the standard `encoding.BinaryMarshaler` interface. Note: This method allocates a new byte slice. For performance-critical paths, use `MarshalTo` or `WriteTo` instead.

func (*Fixed[Payload]) MarshalTo 

func (c *Fixed[Payload]) MarshalTo(p []byte) (int, error)

MarshalTo marshals the struct into the provided slice `p`. This is the most performant marshalling option as it avoids memory allocation.

func (*Fixed[Payload]) ReadFrom 

func (c *Fixed[Payload]) ReadFrom(r io.Reader) (int64, error)

ReadFrom implements `io.ReaderFrom` for efficient, allocation-free reading directly from a stream into the struct.

func (*Fixed[Payload]) Size 

func (c *Fixed[Payload]) Size() int

Size returns the fixed size of the struct in bytes. The result is cached to avoid reflection overhead on subsequent calls.

func (*Fixed[Payload]) UnmarshalBinary 

func (c *Fixed[Payload]) UnmarshalBinary(data []byte) error

UnmarshalBinary implements the standard `encoding.BinaryUnmarshaler` interface. It calls `CheckTrailingNotZeros` to prevent bugs from truncated or oversized payloads.

func (*Fixed[Payload]) WriteTo 

func (c *Fixed[Payload]) WriteTo(w io.Writer) (int64, error)

WriteTo implements `io.WriterTo` for efficient, allocation-free writing directly to a stream (e.g., a network connection or file).

type LimitedReader 

type LimitedReader struct {
	*io.LimitedReader
}

func (*LimitedReader) Close 

func (r *LimitedReader) Close() error

Close closes the underlying reader if it implements io.Closer.

func (*LimitedReader) WriteTo 

func (r *LimitedReader) WriteTo(w io.Writer) (n int64, err error)

WriteTo implements the io.WriterTo interface, providing an optimized path for io.Copy. It reads from the underlying reader, writes to the destination writer

type List 

type List interface {
	Codec
	Len() int
	Codecs() []Codec
}

type List0 

type List0[T Codec] struct {
	// contains filtered or unexported fields
}

func NewList0 

func NewList0[T Codec](items []T) *List0[T]

no Align type List

func (*List0) Codecs 

func (l *List0) Codecs() []Codec

func (*List0) Len 

func (l *List0) Len() int

func (*List0) MarshalBinary 

func (l *List0) MarshalBinary() ([]byte, error)

func (*List0) MarshalTo 

func (l *List0) MarshalTo(buf []byte) (int, error)

func (*List0) ReadFrom 

func (l *List0) ReadFrom(reader io.Reader) (int64, error)

ReadFrom reads and decodes items into the list from a reader. The read behavior is determined by the capacity of the `l.Items` slice: - If cap(l.Items) > 0, it reads exactly that many items. - If cap(l.Items) == 0, it reads items until the reader returns io.EOF.

func (*List0) Size 

func (l *List0) Size() int

Size calculates the total binary size of the list, including alignment padding.

func (*List0) UnmarshalBinary 

func (l *List0) UnmarshalBinary(data []byte) error

func (*List0) WriteTo 

func (l *List0) WriteTo(writer io.Writer) (int64, error)

WriteTo efficiently writes the entire list to a writer, handling alignment.

type List4 

type List4[T Codec] struct {
	// contains filtered or unexported fields
}

func NewList4 

func NewList4[T Codec](items []T) *List4[T]

4 Align type List

func (*List4) Codecs 

func (l *List4) Codecs() []Codec

func (*List4) Len 

func (l *List4) Len() int

func (*List4) MarshalBinary 

func (l *List4) MarshalBinary() ([]byte, error)

func (*List4) MarshalTo 

func (l *List4) MarshalTo(buf []byte) (int, error)

func (*List4) ReadFrom 

func (l *List4) ReadFrom(reader io.Reader) (int64, error)

ReadFrom reads and decodes items into the list from a reader. The read behavior is determined by the capacity of the `l.Items` slice: - If cap(l.Items) > 0, it reads exactly that many items. - If cap(l.Items) == 0, it reads items until the reader returns io.EOF.

func (*List4) Size 

func (l *List4) Size() int

Size calculates the total binary size of the list, including alignment padding.

func (*List4) UnmarshalBinary 

func (l *List4) UnmarshalBinary(data []byte) error

func (*List4) WriteTo 

func (l *List4) WriteTo(writer io.Writer) (int64, error)

WriteTo efficiently writes the entire list to a writer, handling alignment.

type List8 

type List8[T Codec] struct {
	// contains filtered or unexported fields
}

func NewList8 

func NewList8[T Codec](items []T) *List8[T]

8 Align type List

func (*List8) Codecs 

func (l *List8) Codecs() []Codec

func (*List8) Len 

func (l *List8) Len() int

func (*List8) MarshalBinary 

func (l *List8) MarshalBinary() ([]byte, error)

func (*List8) MarshalTo 

func (l *List8) MarshalTo(buf []byte) (int, error)

func (*List8) ReadFrom 

func (l *List8) ReadFrom(reader io.Reader) (int64, error)

ReadFrom reads and decodes items into the list from a reader. The read behavior is determined by the capacity of the `l.Items` slice: - If cap(l.Items) > 0, it reads exactly that many items. - If cap(l.Items) == 0, it reads items until the reader returns io.EOF.

func (*List8) Size 

func (l *List8) Size() int

Size calculates the total binary size of the list, including alignment padding.

func (*List8) UnmarshalBinary 

func (l *List8) UnmarshalBinary(data []byte) error

func (*List8) WriteTo 

func (l *List8) WriteTo(writer io.Writer) (int64, error)

WriteTo efficiently writes the entire list to a writer, handling alignment.

type Marshaler 

type Marshaler interface {
	// encoding.BinaryMarshaler provides the primary encoding method.
	// It allocates and returns a new byte slice.
	encoding.BinaryMarshaler // Method: MarshalBinary() ([]byte, error)
	// io.WriterTo provides efficient, stream-based writing.
	// This avoids allocating the entire byte slice in memory at once.
	io.WriterTo // Method: WriteTo(writer io.Writer) (int64, error)

	// MarshalTo is a high-performance, zero-allocation encoding method.
	// It encodes the object into a pre-allocated buffer, returning an error
	// (e.g., io.ErrShortBuffer) if the buffer is too small.
	MarshalTo(buf []byte) (int, error)
}

Marshaler defines the core methods for encoding an object into a byte stream. It integrates standard library interfaces and provides a high-performance, allocation-free option.

type PeekableReader 

type PeekableReader struct {
	R io.Reader // The underlying reader.
	B []byte    // The buffer for peeked data.
}

PeekableReader is a reader that allows peeking ahead at the underlying data stream.

func PeekReader 

func PeekReader(r io.Reader) *PeekableReader

PeekReader returns a PeekableReader. If the given reader is already a PeekableReader, it is returned directly.

func (*PeekableReader) Close 

func (r *PeekableReader) Close() error

Close closes the underlying reader if it implements io.Closer.

func (*PeekableReader) Peek 

func (r *PeekableReader) Peek(n int) ([]byte, error)

Peek returns the next n bytes without advancing the reader.

func (*PeekableReader) Read 

func (r *PeekableReader) Read(p []byte) (n int, err error)

Read reads data into p. It first reads from the peeked buffer and then from the underlying reader if necessary.

func (*PeekableReader) WriteTo 

func (r *PeekableReader) WriteTo(w io.Writer) (n int64, err error)

WriteTo writes data to w. It first writes the peeked buffer and then copies from the underlying reader.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader provides a buffered reader that simplifies reading binary data. It wraps bufio.Reader and tracks the first error. Subsequent reads become no-ops.

func NewReader 

func NewReader(r io.Reader) (*Reader, error)

NewReader creates a new Reader with a default buffer size.

func NewReaderSize 

func NewReaderSize(r io.Reader, size int) (*Reader, error)

NewReaderSize creates a new Reader with a specified buffer size.

func (*Reader) Align 

func (r *Reader) Align(n int)

Align discard bytes until offset algin with give n.

func (*Reader) Close 

func (r *Reader) Close() error

Close closes the underlying reader if it implements io.Closer.

func (*Reader) Count 

func (r *Reader) Count() int64

func (*Reader) Err 

func (r *Reader) Err() error

func (*Reader) IsEOF 

func (r *Reader) IsEOF() bool

func (*Reader) Read 

func (r *Reader) Read(p []byte) (int, error)

Read implements the io.Reader interface.

func (*Reader) ReadBool 

func (r *Reader) ReadBool(dest *bool)

func (*Reader) ReadByte 

func (r *Reader) ReadByte() (byte, error)

func (*Reader) ReadBytes 

func (r *Reader) ReadBytes(n int) []byte

ReadBytes reads n bytes and returns a new byte slice.

func (*Reader) ReadBytesTo 

func (r *Reader) ReadBytesTo(dest []byte)

func (*Reader) ReadInt16 

func (r *Reader) ReadInt16(dest *int16)

func (*Reader) ReadInt32 

func (r *Reader) ReadInt32(dest *int32)

func (*Reader) ReadInt64 

func (r *Reader) ReadInt64(dest *int64)

func (*Reader) ReadInt8 

func (r *Reader) ReadInt8(dest *int8)

func (*Reader) ReadTo 

func (r *Reader) ReadTo(w io.ReaderFrom)

ReadTo reads data from this reader into an io.ReaderFrom.

func (*Reader) ReadUint16 

func (r *Reader) ReadUint16(dest *uint16)

func (*Reader) ReadUint32 

func (r *Reader) ReadUint32(dest *uint32)

func (*Reader) ReadUint64 

func (r *Reader) ReadUint64(dest *uint64)

func (*Reader) ReadUint8 

func (r *Reader) ReadUint8(dest *uint8)

func (*Reader) Result 

func (r *Reader) Result() (int64, error)

Result returns the total bytes read and the final error state.

func (*Reader) Seek 

func (r *Reader) Seek(offset int64, whence int) (int64, error)

Seek moves the read pointer.

func (*Reader) Size 

func (r *Reader) Size() int

func (*Reader) WithByteOrder 

func (r *Reader) WithByteOrder(order binary.ByteOrder) *Reader

WithByteOrder allows setting a custom byte order and returns the configured for chaining.

func (*Reader) WriteTo 

func (r *Reader) WriteTo(w io.Writer) (int64, error)

WriteTo implements io.WriterTo for efficient copying.

type ReaderPro 

type ReaderPro interface {
	io.ByteReader
	io.Seeker
	Size() int
	// contains filtered or unexported methods
}

type Sizer 

type Sizer interface {
	// Size returns the size of the type in bytes when binary encoded.
	Size() int
}

Sizer is an interface for types that can report their binary size. This is useful for pre-allocating buffers before encoding.

type Unmarshaler 

type Unmarshaler interface {
	// encoding.BinaryUnmarshaler decodes data from a byte slice.
	encoding.BinaryUnmarshaler // Method: UnmarshalBinary(data []byte) error
	// io.ReaderFrom provides efficient, stream-based reading.
	io.ReaderFrom // Method: ReadFrom(r io.Reader) (int64, error)
}

Unmarshaler defines the core methods for decoding a byte stream into an object.

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

Writer provides a buffered writer that simplifies writing binary data. It wraps bufio.Writer for efficiency and tracks the first error that occurs. After an error, all subsequent write operations become no-ops.

func NewWriter 

func NewWriter(w io.Writer) (*Writer, error)

NewWriter creates a new Writer with a default buffer size.

func NewWriterSize 

func NewWriterSize(w io.Writer, size int) (*Writer, error)

NewWriterSize creates a new Writer with a specified buffer size. It returns an error to prevent double-buffering, a common source of bugs.

func (*Writer) Align 

func (w *Writer) Align(n int)

Align write zero bytes until offset algin with give n.

func (*Writer) Close 

func (w *Writer) Close() error

Close closes the underlying writer if it implements io.Closer.

func (*Writer) Count 

func (w *Writer) Count() int64

func (*Writer) Err 

func (w *Writer) Err() error

func (*Writer) Flush 

func (w *Writer) Flush() error

Flush writes any buffered data to the underlying io.Writer.

func (*Writer) ReadFrom 

func (w *Writer) ReadFrom(r io.Reader) (int64, error)

ReadFrom implements io.ReaderFrom for efficient copying.

func (*Writer) Result 

func (w *Writer) Result() (int64, error)

Result flushes the buffer and returns the final count and error state.

func (*Writer) Size 

func (w *Writer) Size() int

func (*Writer) WithByteOrder 

func (w *Writer) WithByteOrder(order binary.ByteOrder) *Writer

WithByteOrder allows setting a custom byte order and returns the configured for chaining.

func (*Writer) Write 

func (w *Writer) Write(buf []byte) (int, error)

Write implements the io.Writer interface.

func (*Writer) WriteBool 

func (w *Writer) WriteBool(v bool)

func (*Writer) WriteByte 

func (w *Writer) WriteByte(v byte) error

func (*Writer) WriteBytes 

func (w *Writer) WriteBytes(buf []byte)

WriteBytes writes a byte slice.

func (*Writer) WriteFrom 

func (w *Writer) WriteFrom(wt io.WriterTo)

WriteFrom reads data from an io.WriterTo.

func (*Writer) WriteInt16 

func (w *Writer) WriteInt16(v int16)

func (*Writer) WriteInt32 

func (w *Writer) WriteInt32(v int32)

func (*Writer) WriteInt64 

func (w *Writer) WriteInt64(v int64)

func (*Writer) WriteInt8 

func (w *Writer) WriteInt8(v int8)

func (*Writer) WriteString 

func (w *Writer) WriteString(str string) (int, error)

Write implements the io.StringWriter interface.

func (*Writer) WriteUint16 

func (w *Writer) WriteUint16(v uint16)

func (*Writer) WriteUint32 

func (w *Writer) WriteUint32(v uint32)

func (*Writer) WriteUint64 

func (w *Writer) WriteUint64(v uint64)

func (*Writer) WriteUint8 

func (w *Writer) WriteUint8(v uint8)

func (*Writer) WriteZeros 

func (w *Writer) WriteZeros(n int64)

WriteZeros writes n zero bytes, often for padding. To avoid allocating a large temporary zero buffer for big padding.

type WriterPro 

type WriterPro interface {
	io.ByteWriter
	io.StringWriter
	Size() int
	Flush() error
	// contains filtered or unexported methods
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.