Your product seems interesting and it can be useful i can see from the features. However, I downloaded your library (all - 1.0.3.jar) to be used locally and even via Maven repo. I faced the following errors when running using Mavan or local:

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class com.landawn.abacus.parser.JSONParserImpl
at com.landawn.abacus.parser.ParserFactory.createJSONParser(ParserFactory.java:194)
at com.landawn.abacus.util.CSVUtil.(CSVUtil.java:69)
at util.ActIDActNameSrcIpTime.TestAbacus.main(TestAbacus.java:24)

The code in TestAbacus.java is below:

`import com.landawn.abacus.DataSet;
import com.landawn.abacus.util.CSVUtil;
import java.io.File;

public class TestAbacus {

public static void main(String args[]) {
    File file = new File("C:\\Projects\\Java\\UserProfile\\data\\Sep_Coumn_1000.csv");
    DataSet dataset = CSVUtil.loadCSV(file);

}
}
`
I used Java jdk1.8.0_141 running on NetBeans IDE 8.1 (Build 201510222201). I hope i can try out your project it seems powerful.

Regards,
KP

It seems this library contains a lot OpenJDK code (e.g. https://github.com/landawn/AbacusUtil/blob/master/src/com/landawn/abacus/util/stream/IntStream.java). OpenJDK is licensed under "GPLv2 with Classpath Exception". This basically forces this project to be GPL2 (with or without classpath exception). I had the same issue with a library of mine and the guys over at opensource stackexchange said:

By including the modified version of the OpenJDK Base64 implementation, your library has to be effectively distributed under the GPLv2 license, with or without the classpath exception.

If you keep the classpath exception, then it is possible to use your library in non-free projects.

If you drop the classpath exception, then any project using your library has to obey the GPLv2 restrictions, which means that the sources must be made available under a GPL-compatible open-source license when the project is distributed.

See: https://opensource.stackexchange.com/questions/7164/i-modified-and-used-gpl2-code-in-my-apache-2-open-source-project-can-my-projec

This basically means, many of your user may be in violation when using without providing their source. This is probably an important thing to note in your main readme, as "Apache2 licensing" is very misleading here.

I have a SQL (as follows) which has few bind variables and one of them is an array (IN clause)

SELECT * from MYTABLE m where m.col1 = ? and m.col2 in (?)

Is there an example which can allow me to do this in SQL Executor?

I found that there are very simple ways to serialize DataSet to csv/json/xml etc. But is there anything that can form a DataSet back again once its sent across the wire. I am looking to transport DataSet directly between components to avoid DTO's entirely.

N.newDataSet or any of the N.from don't seem to support this. I am thinking this must be there somewhere and I am not looking at the right place.

Hi, i have been trying to use the library for the past few days but I have been having a hard time due to limited documentation on the API usage. For example my primary purpose is to use Dataset through JdbcUtil and then apply, join/filter/merge etc on the resulting Dataset.

I guess reading from the DB and populating the Dataset was quick and easy. I was even able to join to Datasets but now I have to

1. Calculate difference between two columns and store the value in a new column
2. Convert the output to a Map with the join key and value as the computed column

I couldnt even figure out how to iterate the Dataset and pull the values from specific columns and populate my pojo list.

Please let me know if there are examples to follow through and understand the usage semantics of the API.

Our SQLExecutor batch inserts are taking a very long time. In the example below for a batch size of 5000 and inserting 19764 rows, it took more than 5 min.

dbSQLExecutor.batchInsert(q.getSql(), SQLExecutor.JdbcSettings.create().setBatchSize(5000)
, dataToBeInserted);

Is there any way to optimize this? We are using default JDBCSettings at this point. Not sure if it supports anything that will help in improving the performance.

From http://www.landawn.com/:

int min = IntList.random(10).distinct().average().getAsInt();

The variable should be called avg or average.

I was trying to figure out, how we can pass in parameter value.
Suppose: i have a query:

"SELECT ID FROM EMPLOYEES WHERE DEP_ID IN ( ? ) and NAME LIKE '%?%' "

DEP_IN :- 1,2,3,4,5,6 and NAME value :- somebody
How can we use SQLExcecutor to run this query?

Was looking to create a dataset with empty rows in them but with column headers so that I can render the grid with an empty dataset. The following results in an exception.

DataSet ds1 = N.newDataSet(N.asList("id", "type", "amount"), null); // NullPointerException

DataSet ds1 = N.newEmptyDataSet(); //Doesn't have the column headers.

Any other way to do this?

Thanks
Andy

DynamoDBExecutor is mentioned in http://www.landawn.com/DynamoDBExecutor_view.html but missing in code base.
Where this class can be found?

I was trying to get to the source for RowDataSet to understand the usage better with respect to fulljoin but couldn't find it in github project.

Is this library, only partially open source?

I couldn't see this in the API but wanted to know if there is away to use bind/named parameters in SQLExecutor?

Hi @landawn

I am trying to perform batch insert but somehow my data is not getting committed in batches.

Like i have passed SQLExecutor.JdbcSetting with batch size as 1000, but if conn is null then it set autoCommit as false and commit data for all batches at once.

can you please suggest something, I need to finish this part urgently.

Thank you in advance.

Sep_Coumn_1000.zip

123

Hi,

I am looking out for something that can create a DataSet from a json String.
For example:

DataSet ds =
                N.newDataSet(
                        N.asLinkedHashMap("ID", N.asList(1, 2),
                                "NAME", N.asList("SAHIL", "NITESH"),
                                "DATA", N.asList("{\"age\": \"27\", \"FATHER_NAME\": \"DHRAM\"}", "{\"age\": \"26\", \"fatherName\": \"PARAM\"}")));

As shown above, my DATA column contains json and now i want to convert this json to a new DataSet.
is there any quick way in API using which i can achieve this.

Actually I am looking for solution urgently, can you please help

Hi @landawn,

I am getting IndexOutOfBoundsException whenever i perform following actions:

1. Joining 2 dataSets out of which 1 dataSet is empty.
   Example:

     final DataSet dataSet1 = N.newDataSet(N.asLinkedHashMap(
             "ACCT_ID", N.asList("1", "2"),
             "ACCT_NM", N.asList(11, 12)));
   
     final DataSet dataSet2= N.newEmptyDataSet(N.asList("ACCT_ID_1","ACCT_NM_1"));
   
     DataSet joinedDataSet = dataSet1.innerJoin(dataSet2, N.asMap("ACCT_ID", "ACCT_ID_1"));
   
   

   2. updating column in case of empty dataSet: 
          **Example:**
       
         final DataSet idDataSet = N.newEmptyDataSet(N.asList("ID"));
         idDataSet.updateColumn("ID", (Try.Function<String, Object, RuntimeException>) Long::valueOf);

I believe this empty check should be part of the API, otherwise as a consumer everybody have to write this check before using above functions.

I have one use-case, where I have dataset and i want to GroupBy on the basis of one column and after grouping by, I want to sort the list of the value for each key.

For example: I have DataSet of Employee with column as ID, NAME, AGE, DEPT_ID?
I want to group by DEP_ID and then sort then by name.

can you please give some example, how can we achieve this with dataset.

Thank you.

Hi @landawn

Just a random question came in my mind after working on your api from last 3 monhts,
This api is no doubt beautifully written, so few question came to my mind, i though i should ask you.

1. What is the though process/idea behind creating this project.
2. How long it took you to write this api.

you worked delicately on this or just to bring the change you developed this beautiful api.

Thank you. have a good day

This is because of a typo in JdbcUtil.
I created a pull request fixing this problem: #15

In my usage of dataset with some federated data sources (Oracle and Postgres) the column names could appear in capitals or lowercase depending on the source DB and its configuration. This becomes a problem when these datasets are joined with each other using their respective column names. For instance "id" fails to join with "ID".

I believe Dataset columns names should be case insensitive so that mismatch doesn't occur during joins or referencing the columns with names.

I would like to be a contributor on this project , found it today and seems very promising and amazing .

Looking on the commit messages to see what happens there is everywhere just release numbers instead of explaining words . This doesn't help future contributors :(

I am having trouble executing queries containing WITH clause or CTE. After debugging I ended up at the following line in SQLExecutor that fails to find any parameters to bind. Tried named and positional parameters. Both resulted in the same.

final int parameterCount = namedSQL.getParameterCount();

Error in case of positional param:
No value specified for parameter 1.' and exception = 'No value specified for parameter 1.'

Error in case of named parameters
PSQLException: ERROR: syntax error at or near ":"

Other SQL's starting with "SELECT" seem to work fine.

My SQL looks something like below, an example structure as the original is much more complex

WITH context as ( SELECT ? as days, ? as curcy FROM DUAL ), transactions as ( SELECT t.* from schema.transactions_tbl t, context c WHERE t.curcy_cd = c.curcy_cd and t.dt <= sysdate - c.days ) SELECT * FROM transactions t WHERE t.status = ''SUCCESS'

I noticed that NamedSQL failed to parse and identify those two bind variables. Am I missing something or the code only works when the SQL starts with SELECT clause.

Hi @landawn

I need your urgent help with below issue.
I am using SQLExecutor, my query looks like this:

"SELECT ID FROM EMPLOYEES WHERE DEP_ID IN (replacMe)"

Now suppose, i want to fetch ids of all employees whose dep_id is 1, 2 or 3.
so i replace string replaceMe with the 3 (?, ?, ?) in the query and query becomes like.

"SELECT ID FROM EMPLOYEES WHERE DEP_ID IN (?, ?, ?)"

When i execute this query it throws error:
SQLException|17041|Missing IN or OUT parameter at index:: 1

can you please help me in this.

The following SQL is an example where we update the properties of json data with the following UPDATE statement

update my_table mt
set data = data || '{"QUANTITY": ?, "AMOUNT": ?, "AMT_LOCL": ?}'
where mt.data ->> 'TRAN_ID'::text = ? and
mt.data ->> 'ACCT_ID'::text = ? and
mt.process_id = ?

com.landawn.abacus.exception.AbacusException: can't mix '?' and '#{propName}' in the same sql script

To me it appears, "::text" is being confused with a named bind parameter.