Comments (6)
Yord
commented on September 28, 2024
1
Thanks for opening this issue @optimbro. Indeed, the behaviour you are describing is a design decision taken by pxi.
Basically, pxi converts each chunk from one format into another. The default chunk is a line (--by line). So each line individually is being transformed into valid JSON with --to json. This is also known as JSON lines format.
pxi was initially written to process files that are several hundred GBs in size in a streaming fashion with minimal memory. This is why it uses the JSON lines format that lends itself to stream processing.
from pxi.
Yord
commented on September 28, 2024
1
Maybe you can achieve what you are looking for with the following?
free -mhw |
pxi "([total, used, free, available]) => ({total, used, free, available})" --from ssv |
awk '{print $0 ","}' |
sed '$ s/.$//' |
cat <(echo "[") - <(echo "]")awk adds commas, sed removes the last comma, cat surrounds the stream with brackets.
from pxi.
Yord
commented on September 28, 2024
1
Thanks for creating this magical stuff, I liked the way I can get and format the output of commands like dh, ls etc 👌😍
🙏 Thank you, I appreciate that.
from pxi.
optimbro
commented on September 28, 2024
1
Maybe you can achieve what you are looking for with the following?
free -mhw | pxi "([total, used, free, available]) => ({total, used, free, available})" --from ssv | awk '{print $0 ","}' | sed '$ s/.$//' | cat <(echo "[") - <(echo "]")
awkadds commas,sedremoves the last comma,catsurrounds the stream with brackets.
Thank you so much ^^, really appreciate it.
from pxi.
optimbro
commented on September 28, 2024
I am not able to use JSON.parse, so I have to do this instead.
let's say
dfCommandResponse = `{"Filesystem":"/dev/sdc","Size":"251G","Used":"7.6G","Avail":"231G","UsePercent":"4%","MountedOn":"/"}
{"Filesystem":"none","Size":"3.4G","Used":"34M","Avail":"3.4G","UsePercent":"1%","MountedOn":"/mnt/wslg"}
{"Filesystem":"none","Size":"3.4G","Used":"4.0K","Avail":"3.4G","UsePercent":"1%","MountedOn":"/mnt/wsl"}
{"Filesystem":"tools","Size":"419G","Used":"104G","Avail":"316G","UsePercent":"25%","MountedOn":"/init"}
{"Filesystem":"none","Size":"3.4G","Used":"0","Avail":"3.4G","UsePercent":"0%","MountedOn":"/dev"}
{"Filesystem":"sysfs","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys"}
{"Filesystem":"proc","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/proc"}
{"Filesystem":"devpts","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/dev/pts"}
{"Filesystem":"none","Size":"3.4G","Used":"8.0K","Avail":"3.4G","UsePercent":"1%","MountedOn":"/run"}
{"Filesystem":"none","Size":"3.4G","Used":"0","Avail":"3.4G","UsePercent":"0%","MountedOn":"/run/lock"}
{"Filesystem":"none","Size":"3.4G","Used":"0","Avail":"3.4G","UsePercent":"0%","MountedOn":"/run/shm"}
{"Filesystem":"none","Size":"3.4G","Used":"0","Avail":"3.4G","UsePercent":"0%","MountedOn":"/run/user"}
{"Filesystem":"binfmt_misc","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/proc/sys/fs/binfmt_misc"}
{"Filesystem":"tmpfs","Size":"3.4G","Used":"0","Avail":"3.4G","UsePercent":"0%","MountedOn":"/sys/fs/cgroup"}
{"Filesystem":"cgroup2","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/unified"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/cpuset"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/cpu"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/cpuacct"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/blkio"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/memory"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/devices"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/freezer"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/net_cls"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/perf_event"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/net_prio"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/hugetlb"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/pids"}
{"Filesystem":"cgroup","Size":"0","Used":"0","Avail":"0","UsePercent":"-","MountedOn":"/sys/fs/cgroup/rdma"}
{"Filesystem":"drivers","Size":"419G","Used":"104G","Avail":"316G","UsePercent":"25%","MountedOn":"/usr/lib/wsl/drivers"}
{"Filesystem":"lib","Size":"419G","Used":"104G","Avail":"316G","UsePercent":"25%","MountedOn":"/usr/lib/wsl/lib"}
{"Filesystem":"none","Size":"3.4G","Used":"76K","Avail":"3.4G","UsePercent":"1%","MountedOn":"/mnt/wslg/versions.txt"}
{"Filesystem":"none","Size":"3.4G","Used":"76K","Avail":"3.4G","UsePercent":"1%","MountedOn":"/mnt/wslg/doc"}
{"Filesystem":"drvfs","Size":"419G","Used":"104G","Avail":"316G","UsePercent":"25%","MountedOn":"/mnt/c"}`
to convert the above to valid json I have to do this below.
const str = `[${dfCommandResponse}]`.split('\n').join(',');
then I am able to parse (below removing last comma)
const jsonStr = JSON.parse(`${str.substring(0, str.length - 2)}]`);
from pxi.
optimbro
commented on September 28, 2024
Thanks @Yord , for the quick response. I got it, it makes sense.
So is there any way like extra command line arguments to get JSON with comma?, else no issues. I'll will go with manual formatting 😊
Thanks for creating this magical stuff, I liked the way I can get and format the output of commands like dh, ls etc 👌😍
from pxi.
Related Issues (7)
- [BUG] Wrong output in example in section 'Usage' HOT 1
- [BUG] pxi-benchmark fails with JavaScript out of heap memory on Gentoo 5.4.6
- Deserializing a JSON HOT 5
- [BUG] Invalid regular expression: /\/: \ at end of pattern HOT 2
- benchmarks are hard to run on Ubuntu HOT 1
- [FEATURE] Please make distinction between jsonl and json and make pxi work with both. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pxi.