FGCOM/jansson
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

introduce new fixed-size key API

Browse Source 
This commit added functions working with fixed-size strings (non null-terminated also).
It's helpful for the following cases:
* getting key from substring without copying to separate buffer (better perfomance)
* using pure UTF-8 keys for the objets
* hack: set binary structs as the keys (see test_binary_keys)

added functions:
 * json_object_getn
 * json_object_setn
 * json_object_setn_nocheck
 * json_object_setn_new
 * json_object_setn_new_nocheck
 * json_object_deln
 * json_object_iter_key_len

added iterators:
 * json_object_keylen_foreach
 * json_object_keylen_foreach_safe

Signed-off-by: Maxim Zhukov <mussitantesmortem@gmail.com>
This commit is contained in:
Maxim Zhukov 2020-01-09 22:18:41 +03:00
parent ba4503804b
commit ca6775dee4
9 changed files with 459 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CMakeLists.txt
View File

@ -513,6 +513,7 @@ if (NOT JANSSON_WITHOUT_TESTS)
test_dump
test_dump_callback
test_equal
test_fixed_size
test_load
test_load_callback
test_loadb

160
doc/apiref.rst
View File

@ -648,6 +648,15 @@ allowed in object keys.
Get a value corresponding to *key* from *object*. Returns *NULL* if
*key* is not found and on error.
.. function:: json_t *json_object_getn(const json_t *object, const char *key, size_t key_len)
.. refcounting:: borrow
Like :func:`json_object_get`, but give the fixed-length *key* with length *key_len*.
See :ref:`fixed_length_keys` for details.
.. versionadded:: 2.14
.. function:: int json_object_set(json_t *object, const char *key, json_t *value)
Set the value of *key* to *value* in *object*. *key* must be a
@ -655,6 +664,13 @@ allowed in object keys.
already is a value for *key*, it is replaced by the new value.
Returns 0 on success and -1 on error.
.. function:: int json_object_setn(json_t *object, const char *key, size_t key_len, json_t *value)
Like :func:`json_object_set`, but give the fixed-length *key* with length *key_len*.
See :ref:`fixed_length_keys` for details.
.. versionadded:: 2.14
.. function:: int json_object_set_nocheck(json_t *object, const char *key, json_t *value)
Like :func:`json_object_set`, but doesn't check that *key* is
@ -662,12 +678,26 @@ allowed in object keys.
really is the case (e.g. you have already checked it by other
means).
.. function:: int json_object_setn_nocheck(json_t *object, const char *key, size_t key_len, json_t *value)
Like :func:`json_object_set_nocheck`, but give the fixed-length *key* with length *key_len*.
See :ref:`fixed_length_keys` for details.
.. versionadded:: 2.14
.. function:: int json_object_set_new(json_t *object, const char *key, json_t *value)
Like :func:`json_object_set()` but steals the reference to
*value*. This is useful when *value* is newly created and not used
after the call.
.. function:: int json_object_setn_new(json_t *object, const char *key, size_t key_len, json_t *value)
Like :func:`json_object_set_new`, but give the fixed-length *key* with length *key_len*.
See :ref:`fixed_length_keys` for details.
.. versionadded:: 2.14
.. function:: int json_object_set_new_nocheck(json_t *object, const char *key, json_t *value)
Like :func:`json_object_set_new`, but doesn't check that *key* is
@ -675,12 +705,26 @@ allowed in object keys.
really is the case (e.g. you have already checked it by other
means).
.. function:: int json_object_setn_new_nocheck(json_t *object, const char *key, size_t key_len, json_t *value)
Like :func:`json_object_set_new_nocheck`, but give the fixed-length *key* with length *key_len*.
See :ref:`fixed_length_keys` for details.
.. versionadded:: 2.14
.. function:: int json_object_del(json_t *object, const char *key)
Delete *key* from *object* if it exists. Returns 0 on success, or
-1 if *key* was not found. The reference count of the removed value
is decremented.
.. function:: int json_object_deln(json_t *object, const char *key, size_t key_len)
Like :func:`json_object_del`, but give the fixed-length *key* with length *key_len*.
See :ref:`fixed_length_keys` for details.
.. versionadded:: 2.14
.. function:: int json_object_clear(json_t *object)
Remove all elements from *object*. Returns 0 on success and -1 if
@ -750,7 +794,7 @@ allowed in object keys.
The items are returned in the order they were inserted to the
object.
**Note:** It's not safe to call ``json_object_del(object, key)``
**Note:** It's not safe to call ``json_object_del(object, key)`` or ``json_object_deln(object, key, key_len)``
during iteration. If you need to, use
:func:`json_object_foreach_safe` instead.
@ -767,11 +811,39 @@ allowed in object keys.
.. function:: void json_object_foreach_safe(object, tmp, key, value)
Like :func:`json_object_foreach()`, but it's safe to call
``json_object_del(object, key)`` during iteration. You need to pass
an extra ``void *`` parameter ``tmp`` that is used for temporary storage.
``json_object_del(object, key)`` or ``json_object_deln(object, key, key_len)`` during iteration.
You need to pass an extra ``void *`` parameter ``tmp`` that is used for temporary storage.
.. versionadded:: 2.8
.. function:: void json_object_keylen_foreach(object, key, key_len, value)
Like :c:func:`json_object_foreach`, but in *key_len* stored length of the *key*.
Example::
/* obj is a JSON object */
const char *key;
json_t *value;
size_t len;
json_object_keylen_foreach(obj, key, len, value) {
printf("got key %s with length %zu\n", key, len);
}
**Note:** It's not safe to call ``json_object_deln(object, key, key_len)``
during iteration. If you need to, use
:func:`json_object_keylen_foreach_safe` instead.
.. versionadded:: 2.14
.. function:: void json_object_keylen_foreach_safe(object, tmp, key, key_len, value)
Like :func:`json_object_keylen_foreach()`, but it's safe to call
``json_object_deln(object, key, key_len)`` during iteration.
You need to pass an extra ``void *`` parameter ``tmp`` that is used for temporary storage.
.. versionadded:: 2.14
The following functions can be used to iterate through all key-value
pairs in an object. The items are returned in the order they were
@ -800,6 +872,12 @@ inserted to the object.
Extract the associated key from *iter*.
.. function:: size_t json_object_iter_key_len(void *iter)
Extract the associated key length from *iter*.
.. versionadded:: 2.14
.. function:: json_t *json_object_iter_value(void *iter)
.. refcounting:: borrow
@ -1909,3 +1987,79 @@ memory, see
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/protect-secrets.html.
The page also explains the :func:`guaranteed_memset()` function used
in the example and gives a sample implementation for it.
.. _fixed_length_keys:
Fixed-Length keys
=================
The Jansson API allows work with fixed-length keys. This can be useful in the following cases:
* The key is contained inside a buffer and is not null-terminated. In this case creating a new temporary buffer is not needed.
* The key contains U+0000 inside it.
List of API for fixed-length keys:
* :c:func:`json_object_getn`
* :c:func:`json_object_setn`
* :c:func:`json_object_setn_nocheck`
* :c:func:`json_object_setn_new`
* :c:func:`json_object_setn_new_nocheck`
* :c:func:`json_object_deln`
* :c:func:`json_object_iter_key_len`
* :c:func:`json_object_keylen_foreach`
* :c:func:`json_object_keylen_foreach_safe`
**Examples:**
Try to write a new function to get :c:struct:`json_t` by path separated by ``.``
This requires:
* string iterator (no need to modify the input for better performance)
* API for working with fixed-size keys
The iterator::
struct string {
const char *string;
size_t length;
};
size_t string_try_next(struct string *str, const char *delimiter) {
str->string += strspn(str->string, delimiter);
str->length = strcspn(str->string, delimiter);
return str->length;
}
#define string_foreach(_string, _delimiter) \
for (; string_try_next(&(_string), _delimiter); (_string).string += (_string).length)
The function::
json_t *json_object_get_by_path(json_t *object, const char *path) {
struct string str;
json_t *out = object;
str.string = path;
string_foreach(str, ".") {
out = json_object_getn(out, str.string, str.length);
if (out == NULL)
return NULL;
}
return out;
}
And usage::
int main(void) {
json_t *obj = json_pack("{s:{s:{s:b}}}", "a", "b", "c", 1);
json_t *c = json_object_get_by_path(obj, "a.b.c");
assert(json_is_true(c));
json_decref(obj);
}

5
src/hashtable.c
View File

@ -322,6 +322,11 @@ void *hashtable_iter_key(void *iter) {
return pair->key;
}
size_t hashtable_iter_key_len(void *iter) {
pair_t *pair = ordered_list_to_pair((list_t *)iter);
return pair->key_len;
}
void *hashtable_iter_value(void *iter) {
pair_t *pair = ordered_list_to_pair((list_t *)iter);
return pair->value;

7
src/hashtable.h
View File

@ -161,6 +161,13 @@ void *hashtable_iter_next(hashtable_t *hashtable, void *iter);
*/
void *hashtable_iter_key(void *iter);
/**
* hashtable_iter_key_len - Retrieve the key length pointed by an iterator
*
* @iter: The iterator
*/
size_t hashtable_iter_key_len(void *iter);
/**
* hashtable_iter_value - Retrieve the value pointed by an iterator
*

5
src/jansson.def
View File

@ -34,9 +34,13 @@ EXPORTS
json_object
json_object_size
json_object_get
json_object_getn
json_object_set_new
json_object_setn_new
json_object_set_new_nocheck
json_object_setn_new_nocheck
json_object_del
json_object_deln
json_object_clear
json_object_update
json_object_update_existing
@ -46,6 +50,7 @@ EXPORTS
json_object_iter_at
json_object_iter_next
json_object_iter_key
json_object_iter_key_len
json_object_iter_value
json_object_iter_set_new
json_object_key_to_iter

33
src/jansson.h
View File

@ -188,9 +188,15 @@ void json_object_seed(size_t seed);
size_t json_object_size(const json_t *object);
json_t *json_object_get(const json_t *object, const char *key)
JANSSON_ATTRS((warn_unused_result));
json_t *json_object_getn(const json_t *object, const char *key, size_t key_len)
JANSSON_ATTRS((warn_unused_result));
int json_object_set_new(json_t *object, const char *key, json_t *value);
int json_object_setn_new(json_t *object, const char *key, size_t key_len, json_t *value);
int json_object_set_new_nocheck(json_t *object, const char *key, json_t *value);
int json_object_setn_new_nocheck(json_t *object, const char *key, size_t key_len,
json_t *value);
int json_object_del(json_t *object, const char *key);
int json_object_deln(json_t *object, const char *key, size_t key_len);
int json_object_clear(json_t *object);
int json_object_update(json_t *object, json_t *other);
int json_object_update_existing(json_t *object, json_t *other);
@ -201,6 +207,7 @@ void *json_object_iter_at(json_t *object, const char *key);
void *json_object_key_to_iter(const char *key);
void *json_object_iter_next(json_t *object, void *iter);
const char *json_object_iter_key(void *iter);
size_t json_object_iter_key_len(void *iter);
json_t *json_object_iter_value(void *iter);
int json_object_iter_set_new(json_t *object, void *iter, json_t *value);
@ -210,6 +217,14 @@ int json_object_iter_set_new(json_t *object, void *iter, json_t *value);
key = json_object_iter_key( \
json_object_iter_next(object, json_object_key_to_iter(key))))
#define json_object_keylen_foreach(object, key, key_len, value) \
for (key = json_object_iter_key(json_object_iter(object)), \
key_len = json_object_iter_key_len(json_object_key_to_iter(key)); \
key && (value = json_object_iter_value(json_object_key_to_iter(key))); \
key = json_object_iter_key( \
json_object_iter_next(object, json_object_key_to_iter(key))), \
key_len = json_object_iter_key_len(json_object_key_to_iter(key)))
#define json_object_foreach_safe(object, n, key, value) \
for (key = json_object_iter_key(json_object_iter(object)), \
n = json_object_iter_next(object, json_object_key_to_iter(key)); \
@ -217,6 +232,14 @@ int json_object_iter_set_new(json_t *object, void *iter, json_t *value);
key = json_object_iter_key(n), \
n = json_object_iter_next(object, json_object_key_to_iter(key)))
#define json_object_keylen_foreach_safe(object, n, key, key_len, value) \
for (key = json_object_iter_key(json_object_iter(object)), \
n = json_object_iter_next(object, json_object_key_to_iter(key)), \
key_len = json_object_iter_key_len(json_object_key_to_iter(key)); \
key && (value = json_object_iter_value(json_object_key_to_iter(key))); \
key = json_object_iter_key(n), key_len = json_object_iter_key_len(n), \
n = json_object_iter_next(object, json_object_key_to_iter(key)))
#define json_array_foreach(array, index, value) \
for (index = 0; \
index < json_array_size(array) && (value = json_array_get(array, index)); \
@ -226,11 +249,21 @@ static JSON_INLINE int json_object_set(json_t *object, const char *key, json_t *
return json_object_set_new(object, key, json_incref(value));
}
static JSON_INLINE int json_object_setn(json_t *object, const char *key, size_t key_len,
json_t *value) {
return json_object_setn_new(object, key, key_len, json_incref(value));
}
static JSON_INLINE int json_object_set_nocheck(json_t *object, const char *key,
json_t *value) {
return json_object_set_new_nocheck(object, key, json_incref(value));
}
static JSON_INLINE int json_object_setn_nocheck(json_t *object, const char *key,
size_t key_len, json_t *value) {
return json_object_setn_new_nocheck(object, key, key_len, json_incref(value));
}
static JSON_INLINE int json_object_iter_set(json_t *object, void *iter, json_t *value) {
return json_object_iter_set_new(object, iter, json_incref(value));
}

49
src/value.c
View File

@ -94,16 +94,32 @@ size_t json_object_size(const json_t *json) {
}
json_t *json_object_get(const json_t *json, const char *key) {
if (!key)
return NULL;
return json_object_getn(json, key, strlen(key));
}
json_t *json_object_getn(const json_t *json, const char *key, size_t key_len) {
json_object_t *object;
if (!key || !json_is_object(json))
return NULL;
object = json_to_object(json);
return hashtable_get(&object->hashtable, key, strlen(key));
return hashtable_get(&object->hashtable, key, key_len);
}
int json_object_set_new_nocheck(json_t *json, const char *key, json_t *value) {
if (!key) {
json_decref(value);
return -1;
}
return json_object_setn_new_nocheck(json, key, strlen(key), value);
}
int json_object_setn_new_nocheck(json_t *json, const char *key, size_t key_len,
json_t *value) {
json_object_t *object;
if (!value)
@ -115,7 +131,7 @@ int json_object_set_new_nocheck(json_t *json, const char *key, json_t *value) {
}
object = json_to_object(json);
if (hashtable_set(&object->hashtable, key, strlen(key), value)) {
if (hashtable_set(&object->hashtable, key, key_len, value)) {
json_decref(value);
return -1;
}
@ -124,22 +140,38 @@ int json_object_set_new_nocheck(json_t *json, const char *key, json_t *value) {
}
int json_object_set_new(json_t *json, const char *key, json_t *value) {
if (!key || !utf8_check_string(key, strlen(key))) {
if (!key) {
json_decref(value);
return -1;
}
return json_object_set_new_nocheck(json, key, value);
return json_object_setn_new(json, key, strlen(key), value);
}
int json_object_setn_new(json_t *json, const char *key, size_t key_len, json_t *value) {
if (!key || !utf8_check_string(key, key_len)) {
json_decref(value);
return -1;
}
return json_object_setn_new_nocheck(json, key, key_len, value);
}
int json_object_del(json_t *json, const char *key) {
if (!key)
return -1;
return json_object_deln(json, key, strlen(key));
}
int json_object_deln(json_t *json, const char *key, size_t key_len) {
json_object_t *object;
if (!key || !json_is_object(json))
return -1;
object = json_to_object(json);
return hashtable_del(&object->hashtable, key, strlen(key));
return hashtable_del(&object->hashtable, key, key_len);
}
int json_object_clear(json_t *json) {
@ -281,6 +313,13 @@ const char *json_object_iter_key(void *iter) {
return hashtable_iter_key(iter);
}
size_t json_object_iter_key_len(void *iter) {
if (!iter)
return 0;
return hashtable_iter_key_len(iter);
}
json_t *json_object_iter_value(void *iter) {
if (!iter)
return NULL;

2
test/suites/api/Makefile.am
View File

@ -7,6 +7,7 @@ check_PROGRAMS = \
test_dump \
test_dump_callback \
test_equal \
test_fixed_size \
test_load \
test_load_callback \
test_loadb \
@ -24,6 +25,7 @@ test_chaos_SOURCES = test_chaos.c util.h
test_copy_SOURCES = test_copy.c util.h
test_dump_SOURCES = test_dump.c util.h
test_dump_callback_SOURCES = test_dump_callback.c util.h
test_fixed_size_SOURCES = test_fixed_size.c util.h
test_load_SOURCES = test_load.c util.h
test_loadb_SOURCES = test_loadb.c util.h
test_memory_funcs_SOURCES = test_memory_funcs.c util.h

205
test/suites/api/test_fixed_size.c Normal file
View File

@ -0,0 +1,205 @@
/*
* Copyright (c) 2020 Petri Lehtinen <petri@digip.org>
*
* Jansson is free software; you can redistribute it and/or modify
* it under the terms of the MIT license. See LICENSE for details.
*/
#include "util.h"
#include <jansson.h>
#include <string.h>
static void test_keylen_iterator(json_t *object) {
const char key1[] = {'t', 'e', 's', 't', '1'};
const char key2[] = {'t', 'e', 's', 't'};
const char key3[] = {'t', 'e', 's', '\0', 't'};
const char key4[] = {'t', 'e', 's', 't', '\0'};
const char *reference_keys[] = {key1, key2, key3, key4};
const size_t reference_keys_len[] = {sizeof(key1), sizeof(key2), sizeof(key3),
sizeof(key4)};
size_t index = 0;
json_t *value;
const char *key;
size_t keylen;
json_object_keylen_foreach(object, key, keylen, value) {
if (keylen != reference_keys_len[index])
fail("invalid key len in iterator");
if (memcmp(key, reference_keys[index], reference_keys_len[index]) != 0)
fail("invalid key in iterator");
index++;
}
}
static void test_keylen(void) {
json_t *obj = json_object();
const char key[] = {'t', 'e', 's', 't', '1'};
const char key2[] = {'t', 'e', 's', 't'};
const char key3[] = {'t', 'e', 's', '\0', 't'};
const char key4[] = {'t', 'e', 's', 't', '\0'};
if (json_object_size(obj) != 0)
fail("incorrect json");
json_object_set_new_nocheck(obj, "test1", json_true());
if (json_object_size(obj) != 1)
fail("incorrect json");
if (json_object_getn(obj, key, sizeof(key)) != json_true())
fail("json_object_getn failed");
if (json_object_getn(obj, key2, sizeof(key2)) != NULL)
fail("false positive json_object_getn by key2");
if (json_object_setn_nocheck(obj, key2, sizeof(key2), json_false()))
fail("json_object_setn_nocheck for key2 failed");
if (json_object_size(obj) != 2)
fail("incorrect json");
if (json_object_get(obj, "test") != json_false())
fail("json_object_setn_nocheck for key2 failed");
if (json_object_getn(obj, key2, sizeof(key2)) != json_false())
fail("json_object_getn by key 2 failed");
if (json_object_getn(obj, key3, sizeof(key3)) != NULL)
fail("false positive json_object_getn by key3");
if (json_object_setn_nocheck(obj, key3, sizeof(key3), json_false()))
fail("json_object_setn_nocheck for key3 failed");
if (json_object_size(obj) != 3)
fail("incorrect json");
if (json_object_getn(obj, key3, sizeof(key3)) != json_false())
fail("json_object_getn by key 3 failed");
if (json_object_getn(obj, key4, sizeof(key4)) != NULL)
fail("false positive json_object_getn by key3");
if (json_object_setn_nocheck(obj, key4, sizeof(key4), json_false()))
fail("json_object_setn_nocheck for key3 failed");
if (json_object_size(obj) != 4)
fail("incorrect json");
test_keylen_iterator(obj);
if (json_object_getn(obj, key4, sizeof(key4)) != json_false())
fail("json_object_getn by key 3 failed");
if (json_object_size(obj) != 4)
fail("incorrect json");
if (json_object_deln(obj, key4, sizeof(key4)))
fail("json_object_deln failed");
if (json_object_getn(obj, key4, sizeof(key4)) != NULL)
fail("json_object_deln failed");
if (json_object_size(obj) != 3)
fail("incorrect json");
if (json_object_deln(obj, key3, sizeof(key3)))
fail("json_object_deln failed");
if (json_object_getn(obj, key3, sizeof(key3)) != NULL)
fail("json_object_deln failed");
if (json_object_size(obj) != 2)
fail("incorrect json");
if (json_object_deln(obj, key2, sizeof(key2)))
fail("json_object_deln failed");
if (json_object_getn(obj, key2, sizeof(key2)) != NULL)
fail("json_object_deln failed");
if (json_object_size(obj) != 1)
fail("incorrect json");
if (json_object_deln(obj, key, sizeof(key)))
fail("json_object_deln failed");
if (json_object_getn(obj, key, sizeof(key)) != NULL)
fail("json_object_deln failed");
if (json_object_size(obj) != 0)
fail("incorrect json");
json_decref(obj);
}
static void test_invalid_keylen(void) {
json_t *obj = json_object();
const char key[] = {'t', 'e', 's', 't', '1'};
json_object_set_new_nocheck(obj, "test1", json_true());
if (json_object_getn(NULL, key, sizeof(key)) != NULL)
fail("json_object_getn on NULL failed");
if (json_object_getn(obj, NULL, sizeof(key)) != NULL)
fail("json_object_getn on NULL failed");
if (json_object_getn(obj, key, 0) != NULL)
fail("json_object_getn on NULL failed");
if (!json_object_setn_new(obj, NULL, sizeof(key), json_true()))
fail("json_object_setn_new with NULL key failed");
if (!json_object_setn_new_nocheck(obj, NULL, sizeof(key), json_true()))
fail("json_object_setn_new_nocheck with NULL key failed");
if (!json_object_del(obj, NULL))
fail("json_object_del with NULL failed");
json_decref(obj);
}
static void test_binary_keys(void) {
json_t *obj = json_object();
int key1 = 0;
int key2 = 1;
json_object_setn_nocheck(obj, (const char *)&key1, sizeof(key1), json_true());
json_object_setn_nocheck(obj, (const char *)&key2, sizeof(key2), json_true());
if (!json_is_true(json_object_getn(obj, (const char *)&key1, sizeof(key1))))
fail("cannot get integer key1");
if (!json_is_true(json_object_getn(obj, (const char *)&key1, sizeof(key2))))
fail("cannot get integer key2");
json_decref(obj);
}
static void test_dump_order(void) {
json_t *obj = json_object();
char key1[] = {'k', '\0', '-', '2'};
char key2[] = {'k', '\0', '-', '1'};
const char expected_sorted_str[] =
"{\"k\\u0000-1\": \"first\", \"k\\u0000-2\": \"second\"}";
const char expected_nonsorted_str[] =
"{\"k\\u0000-2\": \"second\", \"k\\u0000-1\": \"first\"}";
char *out;
json_object_setn_new_nocheck(obj, key1, sizeof(key1), json_string("second"));
json_object_setn_new_nocheck(obj, key2, sizeof(key2), json_string("first"));
out = malloc(512);
json_dumpb(obj, out, 512, 0);
if (memcmp(expected_nonsorted_str, out, sizeof(expected_nonsorted_str) - 1) != 0)
fail("preserve order failed");
json_dumpb(obj, out, 512, JSON_SORT_KEYS);
if (memcmp(expected_sorted_str, out, sizeof(expected_sorted_str) - 1) != 0)
fail("utf-8 sort failed");
free(out);
json_decref(obj);
}
static void run_tests() {
test_keylen();
test_invalid_keylen();
test_binary_keys();
test_dump_order();
}