Differenze

Queste sono le differenze tra la revisione selezionata e la versione attuale della pagina.

--- jsaone [2013/12/14 14:39] – pietro
+++ jsaone [2019/02/05 18:16] (versione attuale) – on github pietro
@@ Linea 1: / Linea 1: @@
-====== jsaone ======
+{{:jsaone:jsaone-0.2.tar.gz|Click here to download version 0.2 of jsaone.}}
-This is a tiny wrapper around the standard json library, allowing to read a json file incrementally.
+{{:jsaone:python-jsaone_0.2-1_amd64.deb|Click here for the (64 bit) Debian/Ubuntu package (Python 2).}}
+{{:jsaone:python3-jsaone_0.2-1_amd64.deb|Click here for the (64 bit) Debian/Ubuntu package (Python 3).}}
+====== What is jsaone? ======
+This is a tiny wrapper around the json module in the Python standard library, allowing to read a json file incrementally.
 This can be useful for
   * parsing json streams without waiting for the end of the transmission,
   * parsing very big json objects without wasting RAM for the json representation itself.
+It is an alternative to [[https://pypi.python.org/pypi/ijson/|ijson]] (written when I did not know ijson existed, but in the end more efficient).
 === Efficiency ===
@@ Linea 13: / Linea 21: @@
 totalling 174 MB:
-^ Parser  ^  Iteration 1 ^ Iteration 2 ^
+^ Parser                          ^  Iteration 1 ^ Iteration 2 ^
-| standard (non-incremental) json |   9.511     |   9.273     |
+| standard (non-incremental) json |    9.511     |    9.273    |
-| cythonized jsaone               |  19.055     |  18.956     |
+| cythonized jsaone               |   19.055     |   18.956    |
-| ijson (with yajl2 backend)      |  62.250     |  64.538     |
+| ijson (with yajl2 backend)      |   62.250     |   64.538    |
-| pure python jsaone              |  421.641     |  421.821     |
+| pure python jsaone              |  421.641     |  421.821    |
 Those results were obtained with the script "**tests/json_load_test.py**".
-Clearly those numbers are affected by the speed of the CPU and of the medium.
+Clearly those numbers are affected by the speed of the CPU and of the medium/stream.
-In general the faster the CPU (compared to the storage medium/stream), the
+In particular, since the test was made on a file from a local hard disk, the
-fastest will be the standard json compared to incremental ones (including
+bottleneck was clearly the CPU, and hence it is disadvantageous for incremental
-jsaone).
+parsers (including jsaone). If the bottleneck is given by the medium/stream,
+jsaone should even outperform the standard json, which will start processing
+only after the entire stream is received.
 === Why "jsaone" ===
@@ Linea 35: / Linea 45: @@
   * [[http://pypi.python.org/pypi/simplejson/|simplejson]] (Python 2.5 only)
-  * for efficiency: [[http://cython.org|cython]] (at build time)
+  * for speedup: [[http://cython.org|cython]] (at build time)
+=== Installing ===
+  - If you use Debian or a derivative (such as Ubuntu or Mint), you can simply use the packages provided above.
+  - **jsaone** is on pypi, so you can install it with //pip install jsaone//
+  - you can extract/clone the git repo, then move in the "jsaone" folder and give the command
+  python3 setup.py build_ext --inplace
+(replace "**python3**" with "**python**" if you are using Python 2).
+=== Usage ===
+  import jsaone
+  with open('/path/to/my/file.json') as f:
+      gen = jsaone.load(f)
+      for key, val in gen:
+          ...
+=== Development ===
+You can browse the git repo [[https://github.com/toobaz/jsaone|here]] or clone with
+  git clone https://github.com/toobaz/jsaone.git
+For bugs and enhancements, just write me - <me@pietrobattiston.it> - ideally pointing to a git branch solving the issue/providing an enhancement.
+Jsaone should be able to parse any compliant json string... so if you find one on which it fails, please let me know!
 === License ===
-Released under the GPL 3.
+Released under the GPL 3. Feel free to contact me if this is a problem for you (and GPL 2 is not).