Comments (10)
Since I haven't heard back in a while, I'm going to close this out. I've refactored json2csv and csv2json, which will address the issue you initially reported. I'm working on updating refactoring the tests and should hopefully have 3.0.0 published to NPM before January 1st.
from json-2-csv.
Hi @NilapuAnusha. Thanks for reporting this. I took a look and this is the output that I'm seeing with the data that you provided:
name,title,description.type,description.count,description.message
ABC,title 1,abc,3,message 1
PQR,title 1,pqr,3,message 1
UVW,title 1,uvw,3,message 1
Here's the code that I'm using to generate that:
let converter = require('json-2-csv');
const data = [
{name: 'ABC', title: 'title 1', description: {type: 'abc', count: 3, message: 'message 1'}},
{name: 'PQR', title: 'title 1', description: {type: 'pqr', count: 3, message: 'message 1'}},
{name: 'UVW', title: 'title 1', description: {type: 'uvw', count: 3, message: 'message 1'}}
];
let json2csvCallback = function (err, csv) {
if (err) throw err;
console.log(csv);
};
converter.json2csv(data, json2csvCallback);
Are you seeing different CSV output? If so, could you please provide me with some code to replicate the issue you're seeing?
Thanks!
from json-2-csv.
Thanks @mrodrig
Description key is object type with varying keys and keys may contain object type values
const data = [
{name: 'ABC', title: 'title 1', description: {type: 'abc', count: 3, message: 'message 1'}},
{name: 'PQR', title: 'title 1', description: {type: 'pqr', count: 3, message: {type: def, count: 5}},
{name: 'UVW', title: 'title 1', description: {type: 'uvw', count: 3, title: "Test"}}
];
Can you let me know the expected output of above one?
from json-2-csv.
@NilapuAnusha Good test case! In that particular scenario, the module will, by default, throw an error because the schemas are different for the different objects: Unhandled rejection Error: Not all documents have the same schema.
However, there is a way around that error. If you specify checkSchemaDifferences: false
in the options object, then the module will convert everything that's available into a CSV and you'll end up with the following result:
name,title,description.type,description.count,description.message,description.message.type,description.message.count,description.title
ABC,title 1,abc,3,message 1,null,null,null
PQR,title 1,pqr,3,def,5,def,5,null
UVW,title 1,uvw,3,null,,,Test
Here's the updated example code:
let converter = require('json-2-csv');
const data = [
{name: 'ABC', title: 'title 1', description: {type: 'abc', count: 3, message: 'message 1'}},
{name: 'PQR', title: 'title 1', description: {type: 'pqr', count: 3, message: {type: 'def', count: 5}}},
{name: 'UVW', title: 'title 1', description: {type: 'uvw', count: 3, title: "Test"}}
];
let json2csvCallback = function (err, csv) {
if (err) throw err;
console.log(csv);
};
converter.json2csv(data, json2csvCallback, {
checkSchemaDifferences: false
});
Hope this helps!
from json-2-csv.
Thanks @mrodrig
But I can see more data columns compared header column and data is messed up(ex: same cell contains two inputs :( )
Could you please check for below example?
[
{"eventId": 185919,
"productId": 1,
"productName": "ABC",
"cId": "10696e00-dcd5-11e8-9075-0e75adsh",
"apiName": "Search",
"userId": 10000001,
"logSource": "get:/search/data",
"logLevel": "Activity",
"logType": "Success",
"dateCreated": "2018-10-31T07:40:01",
"processingTimeMs": 309,
"clientAddress": "111.00.00.000",
"message": {type: "Search", countries: "nl", query:{}}
},
{
"eventId": 185920,
"productId": 1,
"productName": "Search",
"cId": "10232e00-dcd5-11e8-9075-0p75adsh",
"apiName": "Search",
"userId": 100000002,
"logSource": "/get:\\/companies:data.xml:113 (HttpException)",
"logLevel": "Activity",
"logType": "Exception",
"dateCreated": "2018-10-31T05:19:46",
"processingTimeMs": 385,
"clientAddress": "111.00.00.000",
"message": {orderType: "Search", countries: "nl", query:{pageSize: "10]", tradeName: "apple", page: "1"}}
},
{
"eventId": 185921,
"productId": 1,
"productName": "Search",
"cId": "10985e00-dcd5-11e8-9075-0p750dsh",
"apiName": "search",
"userId": 100000005,
"logSource": "/get:\\/companies:data.xml:113 (HttpException)",
"logLevel": "Activity",
"logType": "Exception",
"dateCreated": "2018-10-31T07:19:55",
"processingTimeMs": 237,
"clientAddress": "111.00.00.000",
"message": {orderType: "Search", countries: "nl", query:{pageSize: "10", tradeName: "apple", page: "1"}}
},
{
"eventId": 185922,
"productId": 1,
"productName": "Search",
"cId": null,
"apiName": "search",
"userId": 1000000008,
"logSource": "get:/companies",
"logLevel": "Activity",
"logType": "Success",
"dateCreated": "2018-10-31T08:20:02",
"processingTimeMs": 585,
"clientAddress": "111.00.00.000",
"message": {orderType: "Search", countries: "nl", query:{tradeName :"apple", name: "apple", pageSize: "10", page: "1", id: null, type: null}, totalResults: 372}
},
{
"eventId": 185923,
"productId": 1,
"productName": "Search",
"cId": "015l5170-eryt-11e8-9075-06yusdb427c2",
"apiName": "search",
"userId": 1000000002,
"logSource": "/get:\\/companies:data.xml:113 (HttpException)",
"logLevel": "Activity",
"logType": "Exception",
"dateCreated": "2018-10-31T08:20:17",
"processingTimeMs": 201,
"clientAddress": "111.00.00.000",
"message": {orderType: "Search", countries: "fr", query: {pageSize: "10", tradeName: "apple", page: "1"}}
},
{
"eventId": 185924,
"productId": 1,
"productName": "Search",
"cId": null,
"apiName": "search",
"userId": 1000000008,
"logSource": "get:/companies",
"logLevel": "Activity",
"logType": "Success",
"dateCreated": "2018-10-31T06:20:31",
"processingTimeMs": 339,
"clientAddress": "111.00.00.000",
"message": {orderType: "Search", countries: "fr", query: {tradeName: "apple", name: "apple", pageSize:"10", page:"1", id:null, type:null}, totalResults:599}
},
{
"eventId": 185925,
"productId": 1,
"productName": "Search",
"cId": "10677e00-dcd5-11e8-9075-0p75adsh",
"apiName": "cs-ggs-p-search",
"userId": 100000008,
"logSource": "/get:\\/companies:data.xml:113 (HttpException)",
"logLevel": "Activity",
"logType": "Exception",
"dateCreated": "2018-10-31T06:20:38",
"processingTimeMs": 120,
"clientAddress": "111.93.14.242",
"message": {orderType: "Search", countries: "fr", query:{pageSize: 10,tradeName:"apple",page:"1"}}
}
]
from json-2-csv.
@NilapuAnusha good point about the last example you provided having more data columns than the header. That is unfortunately what the module will currently provide by default since I haven't gotten around to defaulting to use quotes to wrap values. Here's some updated output and example code for that last example:
"name","title","description.type","description.count","description.message","description.message.type","description.message.count","description.title"
"ABC","title 1","abc","3","message 1","null","null","null"
"PQR","title 1","pqr","3",""def","5"","def","5","null"
"UVW","title 1","uvw","3","null","","","Test"
converter.json2csv(data, json2csvCallback, {
checkSchemaDifferences: false,
delimiter: { wrap : '"' }
});
Using the quote wrap option, it'll output:
"eventId","productId","productName","cId","apiName","userId","logSource","logLevel","logType","dateCreated","processingTimeMs","clientAddress","message.type","message.countries","message.query","message.orderType","message.query.pageSize","message.query.tradeName","message.query.page","message.query.name","message.query.id","message.query.type","message.totalResults"
"185919","1","ABC","10696e00-dcd5-11e8-9075-0e75adsh","Search","10000001","get:/search/data","Activity","Success","2018-10-31T07:40:01","309","111.00.00.000","Search","nl","","null","null","null","null","null","null","null","null"
"185920","1","Search","10232e00-dcd5-11e8-9075-0p75adsh","Search","100000002","/get:\/companies:data.xml:113 (HttpException)","Activity","Exception","2018-10-31T05:19:46","385","111.00.00.000","null","nl",""10]","apple","1"","Search","10]","apple","1","null","null","null","null"
"185921","1","Search","10985e00-dcd5-11e8-9075-0p750dsh","search","100000005","/get:\/companies:data.xml:113 (HttpException)","Activity","Exception","2018-10-31T07:19:55","237","111.00.00.000","null","nl",""10","apple","1"","Search","10","apple","1","null","null","null","null"
"185922","1","Search","","search","1000000008","get:/companies","Activity","Success","2018-10-31T08:20:02","585","111.00.00.000","null","nl",""apple","apple","10","1","",""","Search","10","apple","1","apple","","","372"
"185923","1","Search","015l5170-eryt-11e8-9075-06yusdb427c2","search","1000000002","/get:\/companies:data.xml:113 (HttpException)","Activity","Exception","2018-10-31T08:20:17","201","111.00.00.000","null","fr",""10","apple","1"","Search","10","apple","1","null","null","null","null"
"185924","1","Search","","search","1000000008","get:/companies","Activity","Success","2018-10-31T06:20:31","339","111.00.00.000","null","fr",""apple","apple","10","1","",""","Search","10","apple","1","apple","","","599"
"185925","1","Search","10677e00-dcd5-11e8-9075-0p75adsh","cs-ggs-p-search","100000008","/get:\/companies:data.xml:113 (HttpException)","Activity","Exception","2018-10-31T06:20:38","120","111.93.14.242","null","fr",""10","apple","1"","Search","10","apple","1","null","null","null","null"
from json-2-csv.
Thanks @mrodrig
When I download same data in file, I got below one
Column data breaks from message.query
from json-2-csv.
Sorry for the delayed response. I'm working on updating the module to v3.0.0 and will take a look at fixing this in that release.
from json-2-csv.
Hi @NilapuAnusha. Since I'm in the process of working on v3.0.0 of the module and just finished up getting the majority of the functionality working with RFC 4180 support, I wanted to send you the CSV output of the last data set for you to check. If you wouldn't mind looking at the generated CSV and verifying that it matches your expected output, I would greatly appreciate it.
Here's the output from my local v3.0.0 json2csv code:
eventId,productId,productName,cId,apiName,userId,logSource,logLevel,logType,dateCreated,processingTimeMs,clientAddress,message.type,message.countries,message.query,message.orderType,message.query.pageSize,message.query.tradeName,message.query.page,message.query.name,message.query.id,message.query.type,message.totalResults
185919,1,ABC,10696e00-dcd5-11e8-9075-0e75adsh,Search,10000001,get:/search/data,Activity,Success,2018-10-31T07:40:01,309,111.00.00.000,Search,nl,{},undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined
185920,1,Search,10232e00-dcd5-11e8-9075-0p75adsh,Search,100000002,/get:\/companies:data.xml:113 (HttpException),Activity,Exception,2018-10-31T05:19:46,385,111.00.00.000,undefined,nl,"{""pageSize"":""10]"",""tradeName"":""apple"",""page"":""1""}",Search,10],apple,1,undefined,undefined,undefined,undefined
185921,1,Search,10985e00-dcd5-11e8-9075-0p750dsh,search,100000005,/get:\/companies:data.xml:113 (HttpException),Activity,Exception,2018-10-31T07:19:55,237,111.00.00.000,undefined,nl,"{""pageSize"":""10"",""tradeName"":""apple"",""page"":""1""}",Search,10,apple,1,undefined,undefined,undefined,undefined
185922,1,Search,null,search,1000000008,get:/companies,Activity,Success,2018-10-31T08:20:02,585,111.00.00.000,undefined,nl,"{""tradeName"":""apple"",""name"":""apple"",""pageSize"":""10"",""page"":""1"",""id"":null,""type"":null}",Search,10,apple,1,apple,null,null,372
185923,1,Search,015l5170-eryt-11e8-9075-06yusdb427c2,search,1000000002,/get:\/companies:data.xml:113 (HttpException),Activity,Exception,2018-10-31T08:20:17,201,111.00.00.000,undefined,fr,"{""pageSize"":""10"",""tradeName"":""apple"",""page"":""1""}",Search,10,apple,1,undefined,undefined,undefined,undefined
185924,1,Search,null,search,1000000008,get:/companies,Activity,Success,2018-10-31T06:20:31,339,111.00.00.000,undefined,fr,"{""tradeName"":""apple"",""name"":""apple"",""pageSize"":""10"",""page"":""1"",""id"":null,""type"":null}",Search,10,apple,1,apple,null,null,599
185925,1,Search,10677e00-dcd5-11e8-9075-0p75adsh,cs-ggs-p-search,100000008,/get:\/companies:data.xml:113 (HttpException),Activity,Exception,2018-10-31T06:20:38,120,111.93.14.242,undefined,fr,"{""pageSize"":10,""tradeName"":""apple"",""page"":""1""}",Search,10,apple,1,undefined,undefined,undefined,undefined
Hopefully that's much closer to the output that you are expecting than the current v2.4.0 json2csv code. I still need to revamp the csv2json functionality in order to get it up to RFC 4180 compliance, which might take a bit (since I'm only able to work on this after work or on the weekends). If you need the new code sooner though, please let me know and I can work to finalize the last few items I need to address in json2csv and then I can look into releasing a "beta" version of the module.
Thanks again for your help to improve the module's functionality!
from json-2-csv.
Version 3.0.0 is available in beta form on NPM as 3.0.0-beta.0
and should be merged at some point soon in #96.
from json-2-csv.
Related Issues (20)
- Can't be use with a stringify object but only with object that fields are strings HOT 1
- CSV string returned prepends numbering to headers HOT 3
- No keyPath was provided HOT 2
- CSV output is incorrect when JSON value is object or null HOT 1
- Problem with list of objects same keys HOT 5
- parseValue not in type definitions
- Prevent CSV Injection
- Headerless CSV HOT 3
- Incorrect parsing of wrapped values HOT 5
- Custom sorting of headers HOT 2
- How to properly change mongoDB timestamp to proper date? HOT 1
- TypeError: Cannot convert undefined or null to object HOT 1
- Private Vulnerability Disclosure HOT 1
- csv2json Allow CSV Files Without Headers HOT 3
- NestedDotKeys with false and 0 are returned as undefined HOT 4
- prependHeader: false, is not working HOT 2
- Paginated conversion ? HOT 1
- "expandArrayObjects" option not behaving as expected HOT 3
- sync version HOT 2
- Incorrect parsing of wrapped values when the new line is right after the starting quote HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from json-2-csv.