README
¶
JSON Trailing Parser
A lightweight, fast Go library for querying deeply nested JSON data structures using an intuitive path-based syntax. Perfect for extracting data from pre-rendered web pages and complex JSON (Array based Trailing) payloads.
❓ How It Works?
The parser uses a unique two-phase traversal mechanism optimized for trailing reference JSON structures:
Phase 1: Sequential Scan
When you query key1.key2, the parser first scans through the root array sequentially until it finds an object containing the key key1.
Phase 2: Array Jump Access
Once the key is found, the parser performs array jump access. The value at key1 is treated as an index reference, allowing the parser to jump directly to that position in the array to retrieve the actual data, it does so for the rest of the remaining tokens.
🎯 Motivation
While scraping a website, I encountered a pre-rendered page with no direct API endpoint to fetch data from. However, I discovered that all the data I needed was embedded in the page as __NUXT_DATA__ JSON. The challenge was that this JSON used a unique trailing reference structure where object keys mapped to array indices.
Traditional JSON parsers couldn't efficiently query this structure, so JSON Trailing Parser was born! This library makes it trivial to extract deeply nested values using simple, readable query paths.
📖 Table of Contents
- Features
- Installation
- Quick Start
- Query Syntax
- Usage Examples
- API Reference
- Error Handling
- Contributing
- License
✨ Features
- 🎯 Intuitive Query Syntax - Use dot notation and array indexing to navigate JSON structures
- 🛡️ Robust Error Handling - Detailed error messages with context
- 🌟 Wildcard Support - Extract multiple array elements with
[*] - 🔐 Escape Sequences - Handle keys with special characters using
:escape token - 📦 Zero Dependencies - Pure Go implementation
📦 Installation
go get github.com/Techzy-Programmer/json-trailing-parser
🚀 Quick Start
package main
import (
"encoding/json"
"fmt"
"github.com/Techzy-Programmer/json-trailing-parser/jtparser"
)
func main() {
// Sample JSON data (trailing reference format)
jsonData := `[
{
"user": 1, // 1 here points to index where the `user` data resides in top level array object
"profile": 2 // same here
},
{
"name": "Alice",
"age": 30,
"email": "[email protected]"
},
{
"avatar": "https://example.com/avatar.jpg",
"bio": "Software Engineer"
}
]`
// Parse JSON into []any
var searchSpace []any
json.Unmarshal([]byte(jsonData), &searchSpace)
// Create parser and query data
parser, qErr := jtparser.NewMonoParser[string]("user.name", searchSpace)
if qErr != nil {
// Handle error
return
}
result, pErr := parser.Parse()
if pErr != nil {
// Handle error
return
}
fmt.Println(*result) // Output: Alice
}
📝 Query Syntax
Basic Object Access
Use dot notation to access nested objects:
user.profile.name
Array Indexing
Use square brackets to access array elements by index:
users[0].name
items[5].price
Wildcard Array Access
Use [*] to extract all elements from an array:
users[*].name // Returns array of all user names
products[*].price // Returns array of all product prices
Escape Sequences
Use : to escape special characters in key names:
app-router-provider::my-resource.sources // Escapes the : in key name (:: -> ":")
data:.value // Escapes the . in key name ("data.value" is one single key name)
item:[0] // Escapes the [ in key name (this results in invalid query)
Escapable Characters: . [ ] : { }
Query Rules
- Queries always start with a key name (no leading
.or array access identifiers) - Array indexing can only be applied to array-type data
- Empty brackets
[]are not allowed - Queries cannot end with
.or have consecutive dots..
💡 Usage Examples
Complex Real-World Scenario
Input Scraped Data:
[
1
{
"$seoContext": 3,
"video-gallery": 5
},
"MovieWeb | The nature's documentary",
{
"video_title": "Amazing Nature Documentary",
"video_description": "Explore the wonders of nature",
"video_duration": "45:30"
},
990743,
{
"images": [
{"path": "thumb1.jpg", "width": 320},
{"path": "thumb2.jpg", "width": 640},
{"path": "thumb3.jpg", "width": 1280}
]
}
]
Multiple Queries:
// Get video title
titleParser, _ := jtparser.NewMonoParser[string](
"$sseoContext.video_title",
searchSpace,
)
title, _ := titleParser.Parse() // ⚠️ Warning: Never ignore errors in real production code
fmt.Println("Title:", *title) // Amazing Nature Documentary
// Get video duration
durationParser, _ := jtparser.NewMonoParser[string](
"$sseoContext.video_duration",
searchSpace,
)
duration, _ := durationParser.Parse()
fmt.Println("Duration:", *duration) // 45:30
// Get all thumbnail paths
thumbsParser, _ := jtparser.NewMonoParser[[]any](
"video-gallery.images[*].path",
searchSpace,
)
thumbs, _ := thumbsParser.Parse()
fmt.Println("Thumbnails:", *thumbs) // [thumb1.jpg thumb2.jpg thumb3.jpg]
📚 API Reference
NewMonoParser[T any](query string, searchSpace []any) (*MonoParser[T], error)
Creates a new parser instance with the specified query and search space.
Parameters:
query- The query string to parsesearchSpace- The JSON data as[]any(usejson.Unmarshal)
Returns:
*MonoParser[T]- Parser instanceerror- Error if query is malformed
Example:
parser, err := jtparser.NewMonoParser[string]("user.name", searchSpace)
if err != nil {
log.Fatal(err)
}
Parse() (*T, error)
Executes the query and returns the result.
Returns:
*T- Pointer to the result of type Terror- Error if path not found or type mismatch
Example:
result, err := parser.Parse()
if err != nil {
log.Fatal(err)
}
fmt.Println(*result)
ChangeQuery(newQuery string) error
Changes the query without creating a new parser instance.
Parameters:
newQuery- The new query string
Returns:
error- Error if new query is malformed
Example:
err := parser.ChangeQuery("user.email")
if err != nil {
log.Fatal(err)
}
result, _ := parser.Parse()
⚠️ Error Handling
The library provides detailed error types for better debugging:
ErrInvalidQuery
Returned when the query syntax is malformed.
type ErrInvalidQuery struct {
Query string
Reason string
}
Example:
invalid query "user..name": consecutive dots found at position 5
ErrKeyNotFound
Returned when a specified key doesn't exist in the object or array.
type ErrKeyNotFound struct {
Key string
Path string
Parent ParentType // "object" or "array"
}
Example:
key "username" not found in object at path "$.user"
ErrOutOfBounds
Returned when an array index is out of range.
type ErrOutOfBounds struct {
Index int
Path string
}
Example:
index 5 at path "$.items" is out of bounds
ErrTypeMismatch
Returned when the data type doesn't match the expected structure.
type ErrTypeMismatch struct {
Path string
Key string
Object any
}
Example:
type mismatch in search space at path $.user for key profile
errored object >> "string_value_found_instead_of_object"
🤝 Contributing
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
- Fork the repository
- Create your feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
🙏 Acknowledgments
- Inspired by the need to parse complex JSON structures from pre-rendered web pages
- Built with ❤️ for the Go community
Made with ☕ by Rishabh Kumar
If this library helped you, consider giving it a ⭐ on GitHub!
Directories