Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs #1221

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

Docs #1221

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/explanations/1-runtime-features.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@


Challenge: Submit an issue or PR with an improvement
3 changes: 3 additions & 0 deletions docs/explanations/2-compile-time-features.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@


Challenge: Submit an issue or PR with an improvement
4 changes: 4 additions & 0 deletions docs/explanations/3-checking-of-correctness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Printing of generated codecs with macro options (implicit val printCodec: CodecMakerConfig.PrintCodec = new CodecMakerConfig.PrintCodec {})
Using Java decompilers (CFR)

Challenge: Find a missing test coverage for some condition or hidden feature and submit an issue (or a PR with a fix)
Empty file.
6 changes: 6 additions & 0 deletions docs/explanations/5-performance-benchmarks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
How to run JVM and Scala.js benchmarks

Synthetic (for focusing on parsing/serialization of different value types and collections) +
Real world format/API samples

Challenge:
3 changes: 3 additions & 0 deletions docs/explanations/6-impact-on-others.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
framework benchmarks
safety improvements in play-json, spray-json, upickle
fast performance feedback for jackson-module-scala (and underlying jackson-core), circe
3 changes: 3 additions & 0 deletions docs/how-tos/1-how-to-double-check-safety-and-correctness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
- Check if some security sensitive options of compile-time and runtime configuration are safe for your use cases
- Print and analyze sources of generated codecs (check number of anonymous classes, size of generated methods, etc.)
- Patch jsoniter-scala sources with some instrumentation, build and publish locally (as example to check number of instantiated codecs)
24 changes: 24 additions & 0 deletions docs/how-tos/2-how-to-squize-more-performance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Tools to check:
- Load test with realistic or synthetic input using async-profiler in different modes (-e cpu, alloc, wall)
- Write JMH benchmarks with realistic or synthetic input using different profilers (gc, perfnorm, perfasm)
- Decompile byte code with CFR decompiler to Java

Options to consider:
- Tune sizes of internal buffers (do not rely on defaults)
- Do pretty printing or use exception stack traces only for debugging
- Use reading and writing from/to pre-allocated sub-arrays
- Use short up to 8 field names per case class (not more than 64 characters in total)
- Use ASCII field and class names
- Turn off checking of field duplication (when parsing on client side)
- Avoid using string types for data that could be immediately parsed to UUID, data-time, and numbers
- Prefer arrays (or immutable arrays in Scala 3) for JSON arrays, especially when they have primitive type values
- Prefer discriminator keys and avoid discriminator fields
- Use JSON arrays for some stable data structures with all required values (2D/3D coordinates, price/quantity, etc.)
- In product types define required fields first, then most frequently used optional fields
- In enumeration and sum-types define most frequently used classes/objects first
- Turn off hex dumps for exceptions
- Generate decoders/encoders only (reduce code size for Scala.js)

Anti-patterns:
- Generation of codecs for intermediate (non-top level) data structures (as an example, overuse of `derives` keyword)
- Generation and wide usage of codecs for primitives, strings, etc.
6 changes: 6 additions & 0 deletions docs/how-tos/3-how-to-reduce-maintenance-costs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Approaches to consider:
- Switch to Scala 3 (safer and faster generation of smaller codecs)
- Use data-structures that follow JSON representation as close as possible (don't forget compile-time configuration during derivation or using annotations)
- Use 2 data models for huge projects with different API versions or 3-rd party data structures (use chimney or ducktape for transformation between them)
- Use GraphQL (caliban) for reach and highly customized requests
- Use Smithy (smithy4s-json) for model first approach with cross-language APIs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Example: api-key signing
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Example: OpenRTB native
1 change: 1 addition & 0 deletions docs/how-tos/6-how-to-validate-schemaless-JSON.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Example: jsontier-scala-examples/example_02.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dijon, jsoniter-scala-circe, play-json-jsoniter
19 changes: 19 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
1. Goals

- Security
- Correctness
- Performance
- Productivity

2. Features

3. Tutorials

4. How-tos

5. API docs
https://www.javadoc.io/doc/com.github.plokhotnyuk.jsoniter-scala

6. Known-issues

7. Blog posts
202 changes: 202 additions & 0 deletions docs/tutorials/1-getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
# Getting started
Let's start our adventure with jsoniter-scala from parsing and serialization of some complex data structure of nested
collections and case classes.

## Prerequisites
In all tutorials we will use [scala-cli](https://scala-cli.virtuslab.org) for running Scala scripts, so you'll need to install it upfront.

You can use any text editor to copy and paste provided code snippets. Please, also, check Scala CLI Cookbooks
documentation if you use [VSCode](https://scala-cli.virtuslab.org/docs/cookbooks/ide/vscode), [Intellij IDEA](https://scala-cli.virtuslab.org/docs/cookbooks/ide/intellij), or [emacs](https://scala-cli.virtuslab.org/docs/cookbooks/ide/emacs) editors.

Support of Scala CLI by IDEs is improving over time, so that [IntelliJ IDEA](https://www.jetbrains.com/idea/) with the latest version of Scala
plugin you can try its [improved support for Scala CLI projects](https://blog.jetbrains.com/scala/2024/11/13/intellij-scala-plugin-2024-3-is-out/#scala-cli).

Latest versions of jsoniter-scala libraries require JDK 11 or above. You can use any of its distributions.

In all tutorials we will start from a file with the following dependencies and imports:

```scala
//> using scala 3
//> using dep "com.github.plokhotnyuk.jsoniter-scala::jsoniter-scala-core::2.31.3"
//> using compileOnly.dep "com.github.plokhotnyuk.jsoniter-scala::jsoniter-scala-macros::2.31.3"

import com.github.plokhotnyuk.jsoniter_scala.core._
import com.github.plokhotnyuk.jsoniter_scala.macros._
```

Please just copy and paste code snippets from the subsequent steps and run by pressing some `Run` button/keystroke in
your IDE or run it using `scala-cli <file_name>` command from the terminal.

Make sure to preserve indentation when copying and pasting because we will use indenting syntax of Scala 3.

## Definition of a complex data structure

Let's imagine that we need to generate a JSON representation of some report for a US shop.

Below is a definition of its data structures and instantiation of some sample data:

```scala
enum Category extends Enum[Category]:
case Electronics, Fashion, HomeGoods

case class Product(id: Long, name: String, category: Category, price: BigDecimal, description: String)

enum OrderStatus extends Enum[OrderStatus]:
case Pending, Shipped, Delivered, Cancelled

case class OrderItem(product: Product, quantity: Int)

case class Order(id: Long, customer: Customer, items: Seq[OrderItem], status: OrderStatus)

case class Customer(id: Long, name: String, email: String, address: Address)

case class Address(street: String, city: String, state: String, zip: String)

enum PaymentMethod:
case CreditCard(cardNumber: Long, validThru: java.time.YearMonth) extends PaymentMethod
case PayPal(id: String) extends PaymentMethod

case class Payment(method: PaymentMethod, amount: BigDecimal, timestamp: java.time.Instant)

case class OrderPayment(order: Order, payment: Payment)

val product1 = Product(
id = 1L,
name = "Apple iPhone 16",
category = Category.Electronics,
price = BigDecimal(999.99),
description = "A high-end smartphone with advanced camera and AI capabilities"
)
val product2 = Product(
id = 2L,
name = "Nike Air Max 270",
category = Category.Fashion,
price = BigDecimal(129.99),
description = "A stylish and comfortable sneaker with a full-length air unit"
)
val product3 = Product(
id = 3L,
name = "KitchenAid Stand Mixer",
category = Category.HomeGoods,
price = BigDecimal(299.99),
description = "A versatile and powerful stand mixer for baking and cooking"
)
val customer1 = Customer(
id = 1L,
name = "John Doe",
email = "[email protected]",
address = Address(
street = "123 Main St",
city = "Anytown",
state = "CA",
zip = "12345"
)
)
val customer2 = Customer(
id = 2L,
name = "Jane Smith",
email = "[email protected]",
address = Address(
street = "456 Elm St",
city = "Othertown",
state = "NY",
zip = "67890"
)
)
val order1 = Order(
id = 1L,
customer = customer1,
items = Seq(
OrderItem(product1, 1),
OrderItem(product2, 2)
),
status = OrderStatus.Pending
)
val order2 = Order(
id = 2L,
customer = customer2,
items = Seq(
OrderItem(product3, 1)
),
status = OrderStatus.Shipped
)
val paymentMethod1 = PaymentMethod.CreditCard(
cardNumber = 1234_5678_9012_3456L,
validThru = java.time.YearMonth.parse("2026-12")
)
val paymentMethod2 = PaymentMethod.PayPal(
id = "[email protected]"
)
val payment1 = Payment(
method = paymentMethod1,
amount = BigDecimal(1259.97),
timestamp = java.time.Instant.parse("2025-01-03T12:30:45Z")
)
val payment2 = Payment(
method = paymentMethod2,
amount = BigDecimal(299.99),
timestamp = java.time.Instant.parse("2025-01-15T19:10:55Z")
)
val orderPayment1 = OrderPayment(
order = order1,
payment = payment1
)
val orderPayment2 = OrderPayment(
order = order2,
payment = payment2
)
val report = Seq(
orderPayment1,
orderPayment2
)
```

## Defining the codec

Now we need to derive a codec for the report type (`List[OrderPayment]` type in our case). We will use
`JsonCodecMaker.make` macros for that:
```scala
given JsonValueCodec[Seq[OrderPayment]] = JsonCodecMaker.make
```

An instance of this codec (also known as a type-class instance) is getting to be visible in the scope of subsequent
calls of parsing and serialization methods.

## Serialization

Now we are ready to serialize the report. For that we need to define some entry point method and call `writeToString`.
We will also print resulting JSON to the system output to see it as an output:

```scala
@main def gettingStarted: Unit =
val json = writeToString(report)
println(json)
```

From this moment you can run the script and see a long JSON string on the screen.

## Parsing

Having the JSON string in memory you can parse it using following lines that should be pasted to the end of the
`gettingStarted` method:
```scala
val parsedReport = readFromString(json)
println(parsedReport)
```

Let's rerun the script and get additionally printed `toString` representation of a report parsed from JSON string.

If something gone wrong you can pick and run [the final version of a script for this tutorial](1-getting-started.scala).

## Challenge
Experiment with the script to use different types of collections instead of `Seq` and try to find any collection type
from the standard Scala library that is not supported by `JsonCodecMaker.make` macros to derive codec for serialization
and parsing.

## Recap
In this tutorial we learned basics for parsing and serialization of complex nested data structures using `scala-cli`.
Having definition of data structures and their instances in memory we need to:
1. Add dependency usage and package imports of `core` and `macros` modules of jsoniter-scala
2. Define `given` type-class instance of `JsonValueCodec` for top-level data structure
3. Call `writeToString` to serialize an instance of complex data structure to JSON representation
4. Call `readFromString` to parse a complex data structure from JSON representation
Loading