googleapis / java-datastore Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Hello! Autosynth couldn't regenerate java-datastore. ๐
Here's the output from running synth.py
:
Cloning into 'working_repo'...
Switched to a new branch 'autosynth'
Cloning into '/tmpfs/tmp/tmprt4_t8z2/googleapis'...
Cloning into '/tmpfs/tmp/tmprt4_t8z2/synthtool'...
Switched to branch 'autosynth-self'
Note: checking out 'b484256c30ac9117eb82688c801966b6ace856df'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at b484256 deps: update dependency com.google.api.grpc:proto-google-common-protos to v1.18.0 (#117)
Note: checking out 'd741cd976975c745d0199987aff0e908b8352992'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at d741cd97 chore: enable gapicv2 for firestore/v1 API
Note: checking out '52638600f387deb98efb5f9c85fec39e82aa9052'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 5263860 build(java): set GOOGLE_CLOUD_PROJECT env for samples/integration tests (#484)
Switched to a new branch 'autosynth-self-2'
2020-05-05 21:31:00 [INFO] Running synthtool
2020-05-05 21:31:00 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2020-05-05 21:31:01,015 synthtool > Executing /tmpfs/src/github/synthtool/working_repo/synth.py.
On branch autosynth-self-2
nothing to commit, working tree clean
2020-05-05 21:31:01,148 synthtool > Ensuring dependencies.
2020-05-05 21:31:01,160 synthtool > Cloning googleapis.
2020-05-05 21:31:01,161 synthtool > Using precloned repo /tmpfs/tmp/tmprt4_t8z2/googleapis
2020-05-05 21:31:01,165 synthtool > Generating code for: //google/datastore/v1:google-cloud-datastore-v1-java.
2020-05-05 21:31:01,275 synthtool > Failed executing bazel build //google/datastore/v1:google-cloud-datastore-v1-java:
Starting local Bazel server and connecting to it...
Server crashed during startup. Now printing /home/kbuilder/.cache/bazel/_bazel_kbuilder/5528ad577e097974c79b31ce121db9ff/server/jvm.out
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000715200000, 247463936, 0) failed; error='Not enough space' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 247463936 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmpfs/tmp/tmprt4_t8z2/googleapis/hs_err_pid20881.log
2020-05-05 21:31:01,276 synthtool > Wrote metadata to synth.metadata.
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
main()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
spec.loader.exec_module(synth_module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/tmpfs/src/github/synthtool/working_repo/synth.py", line 52, in <module>
proto_path=f'google/{service}/{version}',
File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 63, in java_library
return self._generate_code(service, version, "java", **kwargs)
File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 177, in _generate_code
shell.run(bazel_run_args)
File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
raise exc
File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
encoding="utf-8",
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', 'build', '//google/datastore/v1:google-cloud-datastore-v1-java']' returned non-zero exit status 37.
2020-05-05 21:31:01 [ERROR] Synthesis failed
HEAD is now at b484256 deps: update dependency com.google.api.grpc:proto-google-common-protos to v1.18.0 (#117)
Switched to branch 'autosynth-self'
Note: checking out 'b484256c30ac9117eb82688c801966b6ace856df'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at b484256 deps: update dependency com.google.api.grpc:proto-google-common-protos to v1.18.0 (#117)
Previous HEAD position was d741cd97 chore: enable gapicv2 for firestore/v1 API
HEAD is now at a3a0bf0f BREAKING_CHANGE: Removing TimeSeriesQueryLanguageCondition as an alert condition type. The condition type is unsupported and unused. It was originally added for the Monitoring Query Language Alpha feature. refactor!: Drop support for TimeSeriesQueryLanguageCondition as an alert condition type.
Previous HEAD position was 5263860 build(java): set GOOGLE_CLOUD_PROJECT env for samples/integration tests (#484)
HEAD is now at ab88356 fix: make .kokoro-autosynth executable (#522)
Switched to a new branch 'autosynth-108'
2020-05-05 21:31:01 [INFO] Running synthtool
2020-05-05 21:31:01 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2020-05-05 21:31:01,605 synthtool > Executing /tmpfs/src/github/synthtool/working_repo/synth.py.
On branch autosynth-108
nothing to commit, working tree clean
2020-05-05 21:31:01,736 synthtool > Ensuring dependencies.
2020-05-05 21:31:01,747 synthtool > Cloning googleapis.
2020-05-05 21:31:01,748 synthtool > Using precloned repo /tmpfs/tmp/tmprt4_t8z2/googleapis
2020-05-05 21:31:01,752 synthtool > Generating code for: //google/datastore/v1:google-cloud-datastore-v1-java.
2020-05-05 21:31:01,862 synthtool > Failed executing bazel build //google/datastore/v1:google-cloud-datastore-v1-java:
Starting local Bazel server and connecting to it...
Server crashed during startup. Now printing /home/kbuilder/.cache/bazel/_bazel_kbuilder/5528ad577e097974c79b31ce121db9ff/server/jvm.out
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000715200000, 247463936, 0) failed; error='Not enough space' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 247463936 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmpfs/tmp/tmprt4_t8z2/googleapis/hs_err_pid20916.log
2020-05-05 21:31:01,863 synthtool > Wrote metadata to synth.metadata.
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
main()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
spec.loader.exec_module(synth_module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/tmpfs/src/github/synthtool/working_repo/synth.py", line 52, in <module>
proto_path=f'google/{service}/{version}',
File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 63, in java_library
return self._generate_code(service, version, "java", **kwargs)
File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 177, in _generate_code
shell.run(bazel_run_args)
File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
raise exc
File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
encoding="utf-8",
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', 'build', '//google/datastore/v1:google-cloud-datastore-v1-java']' returned non-zero exit status 37.
2020-05-05 21:31:01 [ERROR] Synthesis failed
HEAD is now at b484256 deps: update dependency com.google.api.grpc:proto-google-common-protos to v1.18.0 (#117)
Switched to branch 'autosynth'
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 584, in <module>
main()
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 465, in main
return _inner_main(temp_dir)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 574, in _inner_main
commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 360, in synthesize_loop
synthesize_inner_loop(toolbox, synthesizer)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 370, in synthesize_inner_loop
synthesizer, len(toolbox.versions) - 1
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 259, in synthesize_version_in_new_branch
synthesizer.synthesize(self.environ)
File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 115, in synthesize
synth_proc.check_returncode() # Raise an exception.
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.
Google internal developers can see the full log here.
Expose the 'reserveIds'
API to Prevents the supplied keys' IDs from being auto-allocated by Cloud Datastore.
/cc: @BenWhitehead
Playing with the native example of Quarkus IO https://quarkus.io/ located at:
https://github.com/quarkusio/quarkus-quickstarts/tree/master/getting-started
Adding a dep with
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-datastore</artifactId>
<version>1.62.0</version>
</dependency>
And using the Cloud Datastore API (simple imports/usage)
Notice I do not define GAE App Engine APIs which I do not want to use (it would not work anyway in a Java11 titanium pure runtime.
I then compile with:
mvn install -Dnative-image.docker-build -Pnative
...
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of Unsafe.objectFieldOffset(Field) is not a constant field., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store.
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of Unsafe.objectFieldOffset(Field) is not a constant field., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store.
[quarkus-quickstart-runner:6] analysis: 22,469.29 ms
Printing call tree to /project/reports/call_tree_quarkus-quickstart-runner_20190312_181219.txt
Printing list of used classes to /project/reports/used_classes_quarkus-quickstart-runner_20190312_181222.txt
Printing list of used packages to /project/reports/used_packages_quarkus-quickstart-runner_20190312_181222.txt
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: com.google.appengine.api.urlfetch.HTTPMethod. To diagnose the issue you can use the --allow-incomplete-classpath option. The missing type is then reported at run time when it is accessed the first time.
Detailed message:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved type during parsing: com.google.appengine.api.urlfetch.HTTPMethod. To diagnose the issue you can use the --allow-incomplete-classpath option. The missing type is then reported at run time when it is accessed the first time.
Trace:
at parsing com.google.api.client.extensions.appengine.http.UrlFetchTransport.buildRequest(UrlFetchTransport.java:116)
Call path from entry point to com.google.api.client.extensions.appengine.http.UrlFetchTransport.buildRequest(String, String):
at com.google.api.client.extensions.appengine.http.UrlFetchTransport.buildRequest(UrlFetchTransport.java:113)
at com.google.api.client.extensions.appengine.http.UrlFetchTransport.buildRequest(UrlFetchTransport.java:50)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:901)
at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:183)
at com.google.datastore.v1.client.Datastore.commit(Datastore.java:87)
at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:152)
at com.google.cloud.datastore.DatastoreImpl$4.call(DatastoreImpl.java:496)
at com.google.cloud.datastore.DatastoreImpl$4.call(DatastoreImpl.java:493)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.datastore.DatastoreImpl.runQuery(DatastoreImpl.java:176)
at com.google.cloud.datastore.QueryResultsImpl.sendRequest(QueryResultsImpl.java:72)
at com.google.cloud.datastore.QueryResultsImpl.computeNext(QueryResultsImpl.java:93)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:140)
at java.util.AbstractCollection.containsAll(AbstractCollection.java:317)
at java.util.RegularEnumSet.containsAll(RegularEnumSet.java:199)
at com.oracle.svm.core.amd64.AMD64CPUFeatureAccess.verifyHostSupportsArchitecture(AMD64CPUFeatureAccess.java:158)
at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:131)
at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
I am reporting an issue I am seeing when using the java-datastore library within an Apache Beam transform (running on GCP Dataflow). The beam job was running slow so i decided to profile it using the GCP Profiler. I was able to identify that most of the wall time was spent waiting on calls to Datastore (within a beam transform) using the Java Datastore library. I wrote a test job that just makes simple datastore reads within a beam transform and saw the same results. Here is a screenshot of the profiler:
You can see in the screenshot that on average it is taking around 19s to run these simple Datastore queries. I understand it is best practice to use the beam datastore library in order to parallelize/batch these calls to Datastore but for our use case we need these sequential synchronous reads (that the Java Datastore library provides) within the beam transform. Anyways I was wondering if anyone had any insight into why these calls might be taking as long as they do? You can see in the screenshot that most of the wall time is spent waiting on the SocketInputStream.socketRead0() method which indicates possible network or remote server latency.
Here is a screenshot of the test job DoFn code i wrote that i ran the above profile on:
For example, the builders for StructuredQuery
don't seem to be tested.
HttpDatastoreRpc.class
private static boolean isLocalHost(String host) { // "::1:8658"
if(!host.isEmpty() {
try {
String normalizedHost = "http://" + removeScheme(host);
InetAddress hostAddr = InetAddress.getByName(new URL(normalizedHost )).getHost()); //throws Exception
return hostAddr.isAnyLocalAddress() || hostAddr.isLoopbackAddress();
} catch (Exception var3) {
}
}
return false;
}
java.lang.IllegalArgumentException: Project endpoint: "::1:8658/v1/projects/my-project" must include scheme.
Note: #97 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.
commit: 964f59e
buildURL: Build Status, Sponge
status: failed
java.lang.AssertionError: expected:<2> but was:<0> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:633) at com.google.cloud.datastore.it.ITDatastoreTest.testSkippedResults(ITDatastoreTest.java:874) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748)
This test failed!
To configure my behavior, see the Build Cop Bot documentation.
If I'm commenting on this issue too often, add the buildcop: quiet
label and
I will stop commenting.
commit: d188c4c
buildURL: Build Status, Sponge
status: failed
com.google.cloud.datastore.DatastoreException: The project java-docs-samples-testing does not exist or it does not contain an active Cloud Datastore or Cloud Firestore database. Please visit http://console.cloud.google.com to create a project or https://console.cloud.google.com/datastore/setup?project=java-docs-samples-testing to add a Cloud Datastore or Cloud Firestore database. Note that Cloud Datastore or Cloud Firestore always have an associated App Engine app and this app must not be disabled. at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:138) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:164) at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:564) at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:561) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.datastore.DatastoreImpl.commit(DatastoreImpl.java:560) at com.google.cloud.datastore.DatastoreImpl.commitMutation(DatastoreImpl.java:553) at com.google.cloud.datastore.DatastoreImpl.delete(DatastoreImpl.java:538) at com.example.datastore.QuickstartSampleIT.deleteTestEntity(QuickstartSampleIT.java:46) at com.example.datastore.QuickstartSampleIT.tearDown(QuickstartSampleIT.java:61) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) Caused by: com.google.datastore.v1.client.DatastoreException: The project java-docs-samples-testing does not exist or it does not contain an active Cloud Datastore or Cloud Firestore database. Please visit http://console.cloud.google.com to create a project or https://console.cloud.google.com/datastore/setup?project=java-docs-samples-testing to add a Cloud Datastore or Cloud Firestore database. Note that Cloud Datastore or Cloud Firestore always have an associated App Engine app and this app must not be disabled., code=NOT_FOUND at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:136) at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:185) at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:96) at com.google.datastore.v1.client.Datastore.commit(Datastore.java:87) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:162) ... 39 more
Hello! Autosynth couldn't regenerate java-datastore. ๐
Here's the output from running synth.py
:
Cloning into 'working_repo'...
Switched to branch 'autosynth'
Running synthtool
['/tmpfs/src/git/autosynth/env/bin/python3', '-m', 'synthtool', 'synth.py', '--']
synthtool > Executing /tmpfs/src/git/autosynth/working_repo/synth.py.
On branch autosynth
nothing to commit, working tree clean
HEAD detached at FETCH_HEAD
nothing to commit, working tree clean
synthtool > Ensuring dependencies.
synthtool > Pulling artman image.
latest: Pulling from googleapis/artman
Digest: sha256:6aec9c34db0e4be221cdaf6faba27bdc07cfea846808b3d3b964dfce3a9a0f9b
Status: Image is up to date for googleapis/artman:latest
synthtool > Cloning googleapis.
synthtool > Running generator for google/datastore/artman_datastore.yaml.
synthtool > Generated code into /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/LookupRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Query.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/GqlQueryParameter.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/QueryOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/LookupResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PropertyOrder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/CompositeFilterOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/FilterOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/QueryResultBatchOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ArrayValueOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/EntityOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/BeginTransactionRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/AllocateIdsResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/KindExpression.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ReserveIdsResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PropertyOrderOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/MutationOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Key.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Projection.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RunQueryRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/GqlQueryParameterOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ProjectionOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/TransactionOptions.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ReadOptionsOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/CommitResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/AllocateIdsRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Value.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RollbackResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Filter.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/LookupResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/CommitRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/TransactionOptionsOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/QueryResultBatch.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RollbackRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ArrayValue.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/AllocateIdsRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/DatastoreProto.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/AllocateIdsResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/BeginTransactionResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ReserveIdsRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/KeyOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Entity.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/Mutation.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/EntityResultOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/LookupRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PartitionId.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/CommitRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PropertyReference.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PropertyReferenceOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/MutationResult.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/GqlQueryOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/CommitResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RunQueryResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PartitionIdOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ReadOptions.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/MutationResultOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/GqlQuery.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RunQueryResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ValueOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ReserveIdsResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/CompositeFilter.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/EntityProto.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/KindExpressionOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RollbackRequestOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RollbackResponse.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/RunQueryRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/EntityResult.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/ReserveIdsRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/BeginTransactionRequest.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PropertyFilter.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/BeginTransactionResponseOrBuilder.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/QueryProto.java.
synthtool > Replaced '// Generated by the protocol buffer compiler. DO NOT EDIT!' in /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java/proto-google-cloud-datastore-v1/src/main/java/com/google/datastore/v1/PropertyFilterOrBuilder.java.
synthtool > Running java formatter on 96 files
synthtool > Running java formatter on 73 files
synthtool > Wrote metadata to synth.metadata.
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/__main__.py", line 102, in <module>
main()
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/__main__.py", line 94, in main
spec.loader.exec_module(synth_module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/tmpfs/src/git/autosynth/working_repo/synth.py", line 66, in <module>
templates = common_templates.java_library()
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/gcp/common.py", line 75, in java_library
return self._generic_library("java_library", **kwargs)
File "/tmpfs/src/git/autosynth/env/lib/python3.6/site-packages/synthtool/gcp/common.py", line 43, in _generic_library
if not kwargs["metadata"]["samples"]:
KeyError: 'samples'
Synthesis failed
Google internal developers can see the full log here.
These tests always timeout when testing with a Firestore in Datastore Mode DB. I think the reason they pass in continuous tests is because the type of DB they use (in project gcloud-devel
) is "Cloud Datastore".
I believe the root cause of the timeouts is the transaction isolation and consistency behavior differences between the 2 Datastore DB types, documented here and here.
TL;DR
With Firestore in datastore mode when 2 concurrent transactions read or write the same data, the lock held by one transaction can delay the other transaction (hence the timeouts), but with Cloud Datastore the contention is reported as an error at commit.
All environments
GOOGLE_CLOUD_PROJECT=some-project GOOGLE_APPLICATION_CREDENTIALS=/tmp/sa-key.json mvn -P enable-integration-tests verify
it.ITDatastoreTest.testTransactionWithQuery
org.junit.runners.model.TestTimedOutException: test timed out after 100 seconds
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:457)
at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
at sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1332)
at sun.security.ssl.SSLSocketImpl.access$300(SSLSocketImpl.java:73)
at sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:948)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:36)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:149)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:97)
at com.google.datastore.v1.client.Datastore.commit(Datastore.java:85)
at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:162)
at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:564)
at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:561)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.datastore.DatastoreImpl.commit(DatastoreImpl.java:560)
at com.google.cloud.datastore.DatastoreImpl.commitMutation(DatastoreImpl.java:553)
at com.google.cloud.datastore.DatastoreImpl.put(DatastoreImpl.java:513)
at com.google.cloud.datastore.DatastoreHelper.put(DatastoreHelper.java:53)
at com.google.cloud.datastore.DatastoreImpl.put(DatastoreImpl.java:488)
at com.google.cloud.datastore.it.ITDatastoreTest.testTransactionWithQuery(ITDatastoreTest.java:284)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
it.ITDatastoreTest.testTransactionWithRead
this test would also timeout if this code was updating ENTITY3 as stated in the comment.
java.lang.AssertionError: Expecting a failure
at org.junit.Assert.fail(Assert.java:89)
at com.google.cloud.datastore.it.ITDatastoreTest.testTransactionWithRead(ITDatastoreTest.java:254)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
.github/workflows/unmanaged_dependency_check.yaml
actions/checkout v4
actions/setup-java v4
datastore-v1-proto-client/pom.xml
com.google.cloud:google-cloud-datastore-parent 2.20.1
com.google.truth:truth 1.4.2
google-cloud-datastore-bom/pom.xml
com.google.cloud:sdk-platform-java-config 3.31.0
com.google.cloud:google-cloud-datastore 2.20.1
com.google.api.grpc:grpc-google-cloud-datastore-admin-v1 2.20.1
com.google.api.grpc:proto-google-cloud-datastore-v1 0.111.1
com.google.api.grpc:proto-google-cloud-datastore-admin-v1 2.20.1
google-cloud-datastore/pom.xml
com.google.cloud:google-cloud-datastore-parent 2.20.1
com.google.guava:guava-testlib 33.2.1-jre
com.google.truth:truth 1.4.2
grpc-google-cloud-datastore-admin-v1/pom.xml
com.google.cloud:google-cloud-datastore-parent 2.20.1
pom.xml
com.google.cloud:sdk-platform-java-config 3.31.0
com.google.api.grpc:proto-google-cloud-datastore-admin-v1 2.20.1
com.google.api.grpc:grpc-google-cloud-datastore-admin-v1 2.20.1
com.google.cloud:google-cloud-datastore 2.20.1
com.google.api.grpc:proto-google-cloud-datastore-v1 0.111.1
com.google.cloud.datastore:datastore-v1-proto-client 2.20.1
junit:junit 4.13.2
org.easymock:easymock 5.2.0
org.apache.maven.plugins:maven-compiler-plugin 3.13.0
com.google.errorprone:error_prone_core 2.28.0
proto-google-cloud-datastore-admin-v1/pom.xml
com.google.cloud:google-cloud-datastore-parent 2.20.1
proto-google-cloud-datastore-v1/pom.xml
com.google.cloud:google-cloud-datastore-parent 2.20.1
samples/install-without-bom/pom.xml
com.google.cloud.samples:shared-configuration 1.2.0
com.google.cloud:google-cloud-datastore 2.20.0
junit:junit 4.13.2
com.google.truth:truth 1.4.2
org.codehaus.mojo:build-helper-maven-plugin 3.6.0
samples/native-image-sample/pom.xml
com.google.cloud.samples:shared-configuration 1.2.0
com.google.cloud:libraries-bom 26.40.0
junit:junit 4.13.2
com.google.truth:truth 1.4.2
org.junit.vintage:junit-vintage-engine 5.10.2
org.graalvm.buildtools:junit-platform-native 0.10.2
org.apache.maven.plugins:maven-surefire-plugin 3.2.5
org.graalvm.buildtools:native-maven-plugin 0.10.2
samples/pom.xml
com.google.cloud.samples:shared-configuration 1.2.0
org.apache.maven.plugins:maven-deploy-plugin 3.1.2
org.sonatype.plugins:nexus-staging-maven-plugin 1.7.0
samples/snapshot/pom.xml
com.google.cloud.samples:shared-configuration 1.2.0
com.google.cloud:google-cloud-datastore 2.20.1
junit:junit 4.13.2
com.google.truth:truth 1.4.2
org.codehaus.mojo:build-helper-maven-plugin 3.6.0
samples/snippets/pom.xml
com.google.cloud.samples:shared-configuration 1.2.0
com.google.cloud:libraries-bom 26.40.0
junit:junit 4.13.2
com.google.truth:truth 1.4.2
.kokoro/presubmit/graalvm-native-17.cfg
com.google.cloud:sdk-platform-java-config 3.31.0
.kokoro/presubmit/graalvm-native.cfg
com.google.cloud:sdk-platform-java-config 3.31.0
.github/workflows/unmanaged_dependency_check.yaml
com.google.cloud:sdk-platform-java-config 3.31.0
This test failed!
To configure my behavior, see the Build Cop Bot documentation.
If I'm commenting on this issue too often, add the buildcop: quiet
label and
I will stop commenting.
commit: 9043ef3
buildURL: Build Status, Sponge
status: failed
I have around 20,000 entities stored in google datastore. The datastore project is in us-central and I am querying from my laptop in California. When I query for all entities and iterate over the results it is taking about 30 seconds. This seems too slow. Am I doing something wrong?
Here is the sample scala code. I am using google-cloud-datastore version 1.105.3 and scala 2.12
import com.google.cloud.datastore._
println(LocalDateTime.now()) //-----> 2020-12-20T10:36:24.242
val q = Query.newEntityQueryBuilder().setKind("Test").build()
val datastore = DatastoreOptions.newBuilder().setProjectId("my-project-id").build().getService
val es = datastore.run(q, Seq.empty[ReadOption]: _*)
println(LocalDateTime.now()) //---> 2020-12-20T10:36:24.881
while (es.hasNext) {
val e = es.next() // Just iterating over the results.
}
println(LocalDateTime.now()) //----> 2020-12-20T10:36:53.870 (~29 seconds later)
The query run returns almost immediately but it seems like the getting the results is taking time.
Also, when I try to get the count of the entities using the suggested method of setting the offset to Int.MaxValue
I always get getSkippedResults
as 1000. I verified that es.hasNext
is false as well.
Right now, https://github.com/GoogleCloudPlatform/google-cloud-java/tree/master/google-cloud-datastore depends on https://github.com/GoogleCloudPlatform/google-cloud-datastore, which is suboptimal for several reasons:
Note: #101 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.
commit: 74f138f
buildURL: Build Status, Sponge
status: failed
java.lang.AssertionError at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at com.google.cloud.datastore.it.ITDatastoreTest.testSetLimit(ITDatastoreTest.java:881) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748)
This test failed!
To configure my behavior, see the Flaky Bot documentation.
If I'm commenting on this issue too often, add the flakybot: quiet
label and
I will stop commenting.
commit: 03b6d5e
buildURL: Build Status, Sponge
status: failed
java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:840) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) at com.google.cloud.testing.BaseEmulatorHelper.sendPostRequest(BaseEmulatorHelper.java:198) at com.google.cloud.datastore.testing.LocalDatastoreHelper.stop(LocalDatastoreHelper.java:300) at com.google.cloud.datastore.testing.ITLocalDatastoreHelperTest.testStartStopResetWithBuilder(ITLocalDatastoreHelperTest.java:185) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
Will there be any support for async non-blocking calls to the datastore?
The following release PRs may have failed:
Hello! Autosynth couldn't regenerate java-datastore. ๐
Here's the output from running synth.py
:
ernal/gapic_generator_python/requirements.txt (line 4))
Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 5))
Using cached https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Saved ./MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting protobuf==3.13.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 6))
Using cached https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Saved ./protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmpfs/tmp/tmp0xmgeb7r/setuptools-tmp/setuptools/__init__.py", line 6, in <module>
import distutils.core
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/_distutils_hack/__init__.py", line 82, in create_module
return importlib.import_module('._distutils', 'setuptools')
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools._distutils'
----------------------------------------
(Command "python setup.py egg_info" failed with error code 1 in /tmpfs/tmp/pip-build-pt2oeod4/pypandoc/
)
ERROR: no such package '@gapic_generator_python_pip_deps//': pip_import failed: Collecting click==7.1.2 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
Saved ./click-7.1.2-py2.py3-none-any.whl
Collecting google-api-core==1.22.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/e0/2d/7c6c75013105e1d2b6eaa1bf18a56995be1dbc673c38885aea31136e9918/google_api_core-1.22.1-py2.py3-none-any.whl
Saved ./google_api_core-1.22.1-py2.py3-none-any.whl
Collecting googleapis-common-protos==1.52.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 3))
Using cached https://files.pythonhosted.org/packages/03/74/3956721ea1eb4bcf7502a311fdaa60b85bd751de4e57d1943afe9b334141/googleapis_common_protos-1.52.0-py2.py3-none-any.whl
Saved ./googleapis_common_protos-1.52.0-py2.py3-none-any.whl
Collecting jinja2==2.11.2 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 4))
Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
Saved ./Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe==1.1.1 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 5))
Using cached https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Saved ./MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting protobuf==3.13.0 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 6))
Using cached https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Saved ./protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting pypandoc==1.5 (from -r /home/kbuilder/.cache/bazel/_bazel_kbuilder/a732f932c2cbeb7e37e1543f189a2a73/external/gapic_generator_python/requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/d6/b7/5050dc1769c8a93d3ec7c4bd55be161991c94b8b235f88bf7c764449e708/pypandoc-1.5.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmpfs/tmp/tmp0xmgeb7r/setuptools-tmp/setuptools/__init__.py", line 6, in <module>
import distutils.core
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/_distutils_hack/__init__.py", line 82, in create_module
return importlib.import_module('._distutils', 'setuptools')
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools._distutils'
----------------------------------------
(Command "python setup.py egg_info" failed with error code 1 in /tmpfs/tmp/pip-build-pt2oeod4/pypandoc/
)
INFO: Elapsed time: 2.107s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
main()
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
spec.loader.exec_module(synth_module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/kbuilder/.cache/synthtool/java-datastore/synth.py", line 52, in <module>
proto_path=f'google/{service}/{version}',
File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 62, in java_library
service, version, "java", tar_strip_components=0, **kwargs
File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_bazel.py", line 183, in _generate_code
shell.run(bazel_run_args)
File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 39, in run
raise exc
File "/tmpfs/src/github/synthtool/synthtool/shell.py", line 33, in run
encoding="utf-8",
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bazel', '--max_idle_secs=240', 'build', '//google/datastore/v1:google-cloud-datastore-v1-java']' returned non-zero exit status 1.
2020-08-30 14:43:44,641 autosynth [ERROR] > Synthesis failed
2020-08-30 14:43:44,642 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 9407c4a docs: update libraries-bom (#211)
2020-08-30 14:43:44,647 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-08-30 14:43:44,652 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 690, in <module>
main()
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 539, in main
return _inner_main(temp_dir)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 670, in _inner_main
commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 375, in synthesize_loop
has_changes = toolbox.synthesize_version_in_new_branch(synthesizer, youngest)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 273, in synthesize_version_in_new_branch
synthesizer.synthesize(synth_log_path, self.environ)
File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 120, in synthesize
synth_proc.check_returncode() # Raise an exception.
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.
Google internal developers can see the full log here.
This test failed!
To configure my behavior, see the Build Cop Bot documentation.
If I'm commenting on this issue too often, add the buildcop: quiet
label and
I will stop commenting.
commit: 32d5e37
buildURL: Build Status, Sponge
status: failed
This test failed!
To configure my behavior, see the Build Cop Bot documentation.
If I'm commenting on this issue too often, add the buildcop: quiet
label and
I will stop commenting.
commit: a695f61
buildURL: Build Status, Sponge
status: failed
java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:840) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) at com.google.cloud.testing.BaseEmulatorHelper.sendPostRequest(BaseEmulatorHelper.java:199) at com.google.cloud.datastore.testing.LocalDatastoreHelper.stop(LocalDatastoreHelper.java:300) at com.google.cloud.datastore.testing.ITLocalDatastoreHelperTest.testStartStopReset(ITLocalDatastoreHelperTest.java:165) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
I use your cloud functions to react some event in topic.
When even has occurred function has to select an entity to change a state.
But it doesn't work. The library doesn't update en entity in Datastore.
Datastore datastore = DatastoreOptions.getDefaultInstance().getService()
Query<Entity> query = Query.newEntityQueryBuilder()
.setKind("MyEntity")
.setFilter(eq("phoneNumber", "+15551014397"))
.build();
QueryResults<Entity> results = datastore.run(query);
Entity entity = results.next();
Transaction transaction = datastore.newTransaction();
Entity updatedEntity = Entity.newBuilder(entity)
.set("state", 3)
.build();
transaction.update(updatedEntity);
transaction.commit();
The result is success. No error.
But the entity in Datastore has not been updated with field entity.state = 3 - there is previous value/
org.springframework.cloud:spring-cloud-gcp-starter-data-datastore:1.2.5.RELEASE
, I checked - same with the latestgoogle-cloud-datastore:1.104.0
google-cloud-core:1.93.7
This version emulator fails to start without --project
property
gcloud beta emulators datastore start
ERROR: (gcloud.beta.emulators.datastore.start) The required property [project] is not currently set.
You may set it for your current workspace by running:
$ gcloud config set project VALUE
or it can be set temporarily by the environment variable [CLOUDSDK_CORE_PROJECT]
Starting emulator with Java code like this:
LocalDatastoreHelper.create(settings.getConsistency(), settings.getPort());
Causes: no messages in logs, the application failed to call the Datastore then with I/O exceptions, i.e. curl doesn't work.
Set env variable: CLOUDSDK_CORE_PROJECT
with the projectId value
LocalDatastoreHelper
doesn't pass --project
on startup and doesn't expect this from the builder?LocalDatastoreHelper #start
fail to start on this kind of errors?Besides the outcomes of these questions, I would expect this class to fail to start or log something.
Is your feature request related to a problem? Please describe.
When attempting to use the Downloadable emulator, LocalDatastoreHelper downloads from a hardcoded URL to the file in the "gcd" GCS bucket:
Describe the solution you'd like
If I could provide the emulator file URL in an environment variable, I could copy the file from the "gcd" bucket into another bucket that is accessible from the restricted environment and use that URL.
Describe alternatives you've considered
N/A
As of grpc-api:1.26.0
and guava:28.1-android
there is a failure for animal-sniffer-annotations
There is no actual difference between the single annotation present in the versions, so it is being excluded for the time being.
The exclusion should be removed from the poms when there is no longer a disparity.
Hello! Autosynth couldn't regenerate java-datastore. ๐
Here's the output from running synth.py
:
Cloning into 'working_repo'...
Switched to a new branch 'autosynth'
Cloning into '/tmpfs/tmp/tmpxgwz8b6o/googleapis'...
Cloning into '/tmpfs/tmp/tmpxgwz8b6o/synthtool'...
Switched to branch 'autosynth-self'
Note: checking out '97864cbbffdb29420745491eaabb212da9b48147'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 97864cb chore(deps): update dependency com.google.cloud:libraries-bom to v5.3.0 (#115)
Note: checking out 'd741cd976975c745d0199987aff0e908b8352992'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at d741cd97 chore: enable gapicv2 for firestore/v1 API
Note: checking out '52638600f387deb98efb5f9c85fec39e82aa9052'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 5263860 build(java): set GOOGLE_CLOUD_PROJECT env for samples/integration tests (#484)
Switched to a new branch 'autosynth-self-2'
2020-04-30 13:59:09 [INFO] Running synthtool
2020-04-30 13:59:09 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 142, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 109, in _get_module_details
__import__(pkg_name)
File "/tmpfs/src/github/synthtool/synthtool/__init__.py", line 21, in <module>
from synthtool import update_check
File "/tmpfs/src/github/synthtool/synthtool/update_check.py", line 19, in <module>
import packaging.version
ModuleNotFoundError: No module named 'packaging'
2020-04-30 13:59:09 [ERROR] Synthesis failed
HEAD is now at 97864cb chore(deps): update dependency com.google.cloud:libraries-bom to v5.3.0 (#115)
Switched to branch 'autosynth-self'
Note: checking out '97864cbbffdb29420745491eaabb212da9b48147'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 97864cb chore(deps): update dependency com.google.cloud:libraries-bom to v5.3.0 (#115)
Previous HEAD position was d741cd97 chore: enable gapicv2 for firestore/v1 API
HEAD is now at 7e1c7603 Add an OAuth scope annotation in build_service proto file
Previous HEAD position was 5263860 build(java): set GOOGLE_CLOUD_PROJECT env for samples/integration tests (#484)
HEAD is now at 6b685a2 fix: synthtool path (#515)
Switched to a new branch 'autosynth-90'
2020-04-30 13:59:09 [INFO] Running synthtool
2020-04-30 13:59:09 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 142, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 109, in _get_module_details
__import__(pkg_name)
File "/tmpfs/src/github/synthtool/synthtool/__init__.py", line 21, in <module>
from synthtool import update_check
File "/tmpfs/src/github/synthtool/synthtool/update_check.py", line 19, in <module>
import packaging.version
ModuleNotFoundError: No module named 'packaging'
2020-04-30 13:59:09 [ERROR] Synthesis failed
HEAD is now at 97864cb chore(deps): update dependency com.google.cloud:libraries-bom to v5.3.0 (#115)
Switched to branch 'autosynth'
Traceback (most recent call last):
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 576, in <module>
main()
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 457, in main
return _inner_main(temp_dir)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 566, in _inner_main
commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 352, in synthesize_loop
synthesize_inner_loop(toolbox, synthesizer)
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 362, in synthesize_inner_loop
synthesizer, len(toolbox.versions) - 1
File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 259, in synthesize_version_in_new_branch
synthesizer.synthesize(self.environ)
File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 115, in synthesize
synth_proc.check_returncode() # Raise an exception.
File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.
Google internal developers can see the full log here.
Below links in the updating-data and Complete source code section of the datastore README result in 404s.
The Datastore Emulator URL in LocalDatastoreHelper no longer seems to work.
It resolves to this link:
http://storage.googleapis.com/gcd/tools/cloud-datastore-emulator-1.2.0.zip
However, we get the error when trying to access it:
<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object.</Details>
</Error>
I am using Spring Cloud GCP. It has the auto-configuration to start datastore emulator when starting the app engine devserver. However, when I read through the set up of LocalDatastoreHelper, I could see that the gcloudCommand is hard-coded. Are there any ways that I can customize the options of gcloudCommand ?
List<String> gcloudCommand = new ArrayList<>(Arrays.asList(GCLOUD_CMD_TEXT.split(" ")));
gcloudCommand.add(GCLOUD_CMD_PORT_FLAG + "localhost:" + getPort());
gcloudCommand.add(CONSISTENCY_FLAG + consistency);
gcloudCommand.add("--no-store-on-disk");
GcloudEmulatorRunner gcloudRunner =
new GcloudEmulatorRunner(gcloudCommand, VERSION_PREFIX, MIN_VERSION);
if (gcdPath != null) {
gcloudCommand.add("--data-dir=" + gcdPath.toString());
}
I want to remove the option --not-store-on-disk
and would like to change the --data-dir
to my local folder.
Moreover, could you help me to find the location of the current data-dir. I do not know where it is.
Thanks
The add(entity)
method in a Transaction
object is not multi-thread safe. Other methods (e.g. put, delete) may also not be safe but we have traced a failure in our server specifically to add
. Our server uses Scala Futures and so uses multiple threads of execution.
The particular scenario we tracked down was in our new user creation processing where we add 4 entities in a transaction. However there was a probability that one of the entities didn't get saved. It wasn't always the same entity kind that "disappeared". We traced this by adding debug logging after the 'add' calls that shows the add
s successfully completing - but subsequent processing would find one of the entities missing. We did note that it always seemed to be just one entity missing and when an entity did go missing it seemed to be the first entity kind to be added (thread execution order was not consistent across invocations).
We added synchronize
on each of our DataStore calls and haven't encountered this issue since. So at a minimum this issue can serve as a warning to others. Maybe a note could be added to the documentation. But we are also patiently waiting for issue #22 (async support) to be implemented and would hope that explicit synchronization to not be needed.
The issue appears every time during test run. Probably it's related to big number of write/read/delete operations but i'm not sure. Please let me know if any other logs are necessary.
[datastore] Nov 26, 2020 4:47:12 PM com.google.cloud.datastore.emulator.impl.LocalDatastoreFileStub <init>
[datastore] INFO: Local Datastore initialized:
[datastore] Type: High Replication
[datastore] Storage: In-memory
[datastore] API endpoint: http://localhost:8081
[datastore] If you are using a library that supports the DATASTORE_EMULATOR_HOST environment variable, run:
[datastore]
[datastore] export DATASTORE_EMULATOR_HOST=localhost:8081
[datastore]
[datastore] Dev App Server is now running.
[datastore]
[datastore] The previous line was printed for backwards compatibility only.
[datastore] If your tests rely on it to confirm emulator startup,
[datastore] please migrate to the emulator health check endpoint (/). Thank you!
[datastore] The health check endpoint for this emulator instance is http://localhost:8081/Nov 26, 2020 4:47:49 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[datastore] INFO: Adding handler(s) to newly registered Channel.
[datastore] Nov 26, 2020 4:47:49 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[datastore] INFO: Detected non-HTTP/2 connection.
[datastore] Nov 26, 2020 4:47:50 PM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[datastore] INFO: Adding handler(s) to newly registered Channel.
[datastore] Nov 26, 2020 4:47:50 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[datastore] INFO: Detected HTTP/2 connection.
[datastore] Nov 26, 2020 4:47:53 PM io.netty.util.ResourceLeakDetector reportTracedLeak
[datastore] SEVERE: LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
[datastore] Recent access records:
[datastore] Created at:
[datastore] io.netty.buffer.AbstractByteBufAllocator.compositeDirectBuffer(AbstractByteBufAllocator.java:221)
[datastore] io.netty.buffer.AbstractByteBufAllocator.compositeBuffer(AbstractByteBufAllocator.java:199)
[datastore] io.netty.handler.codec.MessageAggregator.decode(MessageAggregator.java:255)
[datastore] io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
[datastore] io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
[datastore] io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
[datastore] io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
[datastore] java.lang.Thread.run(Thread.java:748)
[datastore] Nov 26, 2020 4:47:53 PM io.netty.util.ResourceLeakDetector reportTracedLeak
[datastore] SEVERE: LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
[datastore] Recent access records:
[datastore] Created at:
[datastore] io.netty.buffer.AbstractByteBufAllocator.compositeDirectBuffer(AbstractByteBufAllocator.java:221)
[datastore] io.netty.buffer.AbstractByteBufAllocator.compositeBuffer(AbstractByteBufAllocator.java:199)
[datastore] io.netty.handler.codec.MessageAggregator.decode(MessageAggregator.java:255)
[datastore] io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:426)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
[datastore] io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
[datastore] io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
[datastore] io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
[datastore] java.lang.Thread.run(Thread.java:748)
[datastore] Nov 26, 2020 4:47:53 PM io.netty.util.ResourceLeakDetector reportTracedLeak
[datastore] SEVERE: LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
[datastore] Recent access records:
[datastore] Created at:
[datastore] io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:331)
[datastore] io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:185)
[datastore] io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:176)
[datastore] io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:137)
[datastore] io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
[datastore] io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:147)
[datastore] io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656)
[datastore] io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:591)
[datastore] io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:508)
[datastore] io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
[datastore] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[datastore] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[datastore] java.lang.Thread.run(Thread.java:748)
[datastore] Nov 26, 2020 4:47:53 PM io.netty.util.ResourceLeakDetector reportTracedLeak
[datastore] SEVERE: LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
[datastore] Recent access records:
[datastore] Created at:
[datastore] io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:331)
[datastore] io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:185)
[datastore] io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:176)
[datastore] io.netty.buffer.CompositeByteBuf.allocBuffer(CompositeByteBuf.java:1762)
[datastore] io.netty.buffer.CompositeByteBuf.copy(CompositeByteBuf.java:1417)
[datastore] io.netty.buffer.AbstractByteBuf.copy(AbstractByteBuf.java:1194)
[datastore] io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler.handle(HttpAdapter.java:517)
[datastore] io.gapi.emulators.grpc.HttpAdapter.handleRequest(HttpAdapter.java:191)
[datastore] io.gapi.emulators.netty.HttpHandler.channelRead(HttpHandler.java:52)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.HttpHandler$1.onValue(HttpHandler.java:64)
[datastore] io.gapi.emulators.netty.HttpHandler$1.onValue(HttpHandler.java:54)
[datastore] io.gapi.emulators.grpc.HttpAdapter.handleRequest(HttpAdapter.java:162)
[datastore] io.gapi.emulators.netty.HttpHandler.channelRead(HttpHandler.java:52)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.UnsupportedApiVersionHandler.channelRead(UnsupportedApiVersionHandler.java:96)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.gapi.emulators.netty.SimpleHttpHandler.channelRead(SimpleHttpHandler.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[datastore] io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
[datastore] io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
[datastore] io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[datastore] io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38)
[datastore] io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353)
[datastore] io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
[datastore] io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:474)
[datastore] io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
[datastore] java.lang.Thread.run(Thread.java:748)
[datastore] *** shutting down gRPC server since JVM is shutting down
[datastore] *** server shut down
[datastore] Nov 26, 2020 4:47:57 PM com.google.cloud.datastore.emulator.impl.BaseLocalDatastore cleanupActiveServices
[datastore] INFO: scheduler shutting down.
The release PR #113 is still in a pending state after several hours
public List<Campaign3> getLiveCampaignsForAccount(Long accountId) {
EntityQueryRequest request = datastoreAdapter.getEm().createEntityQueryRequest("SELECT * FROM " + ENTITY_NAME + " WHERE accountId=@1 and live = @2");
request.addPositionalBinding(accountId);
request.addPositionalBinding(true);
QueryResponse<Campaign3> response = datastoreAdapter.getEm().executeEntityQueryRequest(Campaign3.class, request);
return response.getResults();
We're in the process of moving for AppEngine Standard to Flex, thus we're now using the Google Cloud Datastore APIs. Unfortunately, these APIs are not using OpenCensus Tracing and Stats to transmit relevant data.
Please consider this as a feature request for parity.
Manual copy from googleapis/google-cloud-datastore#248
@alysyk please follow along here for progress/work related to your request.
This test failed!
To configure my behavior, see the Flaky Bot documentation.
If I'm commenting on this issue too often, add the flakybot: quiet
label and
I will stop commenting.
commit: 947befe
buildURL: Build Status, Sponge
status: failed
com.google.cloud.datastore.DatastoreException: I/O error at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:138) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:164) at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:564) at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:561) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.datastore.DatastoreImpl.commit(DatastoreImpl.java:560) at com.google.cloud.datastore.DatastoreImpl.commitMutation(DatastoreImpl.java:553) at com.google.cloud.datastore.DatastoreImpl.delete(DatastoreImpl.java:538) at com.example.datastore.QuickstartSampleIT.deleteTestEntity(QuickstartSampleIT.java:46) at com.example.datastore.QuickstartSampleIT.tearDown(QuickstartSampleIT.java:61) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:364) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:237) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548) Caused by: com.google.datastore.v1.client.DatastoreException: I/O error, code=UNAVAILABLE at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:136) at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:105) at com.google.datastore.v1.client.Datastore.commit(Datastore.java:87) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:162) ... 39 more Caused by: java.io.IOException: Error getting access token for service account: 400 Bad Request POST https://oauth2.googleapis.com/token {"error":"invalid_grant","error_description":"Invalid JWT Signature."} at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:612) at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:164) at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:149) at com.google.auth.oauth2.ServiceAccountCredentials.getRequestMetadata(ServiceAccountCredentials.java:946) at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:91) at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:159) at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:109) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc$1.initialize(HttpDatastoreRpc.java:91) at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:91) ... 41 more Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request POST https://oauth2.googleapis.com/token {"error":"invalid_grant","error_description":"Invalid JWT Signature."} at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1116) at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:609) ... 49 more
beta
component: gcloud components remove beta
LocalDatastoreHelper
import com.google.cloud.datastore.testing.LocalDatastoreHelper;
public class Test {
public static void main(String[] args) throws Exception {
LocalDatastoreHelper emulator = LocalDatastoreHelper.create();
emulator.start();
System.out.println("Started!");
}
}
beta
componentroot@17224e8a6010:/app# gcloud components remove beta --quiet
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ These components will be removed. โ
โโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโฌโโโโโโโค
โ Name โ Version โ Size โ
โโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโผโโโโโโโค
โ gcloud Beta Commands โ 2019.05.17 โ โ
โโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโ
...
Uninstall done!
root@17224e8a6010:/app# mvn exec:java -Dexec.mainClass=Test
Ctrl+Z
and run ps
to find out what command is executed via LocalDatastoreHelper^Z
[1]+ Stopped mvn exec:java -Dexec.mainClass=Test
root@17224e8a6010:/app# ps -ef
...
root 1840 1795 0 17:40 pts/1 00:00:00 python2 -S /root/google-cloud-sdk/lib/gcloud.py beta emulators datastore start --host-port=localhost:40079 --consistency=0.9
beta
componentroot@17224e8a6010:/app# python2 -S /root/google-cloud-sdk/lib/gcloud.py beta emulators datastore start --host-port=localhost:40079 --consistency=0.9
You do not currently have this command group installed. Using it
requires the installation of components: [beta]
Your current Cloud SDK version is: 322.0.0
Installing components from version: 322.0.0
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ These components will be installed. โ
โโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโฌโโโโโโโโโโค
โ Name โ Version โ Size โ
โโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโผโโโโโโโโโโค
โ gcloud Beta Commands โ 2019.05.17 โ < 1 MiB โ
โโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโ
For the latest full release notes, please visit:
https://cloud.google.com/sdk/release_notes
Do you want to continue (Y/n)?
Add --quiet
option to run emulator, so as to silently install beta
component.
pomu0325@e2cb8f2
The release PR #278 is still in a pending state after several hours
This test failed!
To configure my behavior, see the Flaky Bot documentation.
If I'm commenting on this issue too often, add the flakybot: quiet
label and
I will stop commenting.
commit: 6822385
buildURL: Build Status, Sponge
status: failed
java.lang.AssertionError at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at com.google.cloud.datastore.it.ITDatastoreTest.testRunStructuredQuery(ITDatastoreTest.java:577) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748)
Follow up to original upgrade proposed in #9
The release PR #113 is still in a pending state after several hours
This test failed!
To configure my behavior, see the Flaky Bot documentation.
If I'm commenting on this issue too often, add the flakybot: quiet
label and
I will stop commenting.
commit: b842b7d
buildURL: Build Status, Sponge
status: failed
java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:840) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) at com.google.cloud.testing.BaseEmulatorHelper.sendPostRequest(BaseEmulatorHelper.java:198) at com.google.cloud.datastore.testing.LocalDatastoreHelper.stop(LocalDatastoreHelper.java:300) at com.google.cloud.datastore.testing.ITLocalDatastoreHelperTest.testStartStopReset(ITLocalDatastoreHelperTest.java:165) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
This test failed!
To configure my behavior, see the Flaky Bot documentation.
If I'm commenting on this issue too often, add the flakybot: quiet
label and
I will stop commenting.
commit: 6822385
buildURL: Build Status, Sponge
status: failed
java.lang.AssertionError: expected:<2> but was:<3> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:633) at com.google.cloud.datastore.it.ITDatastoreTest.testSkippedResults(ITDatastoreTest.java:874) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748)
The following release PRs may have failed:
This test failed!
To configure my behavior, see the Flaky Bot documentation.
If I'm commenting on this issue too often, add the flakybot: quiet
label and
I will stop commenting.
commit: 947befe
buildURL: Build Status, Sponge
status: failed
com.google.cloud.datastore.DatastoreException: I/O error at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.translate(HttpDatastoreRpc.java:138) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:164) at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:564) at com.google.cloud.datastore.DatastoreImpl$5.call(DatastoreImpl.java:561) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.datastore.DatastoreImpl.commit(DatastoreImpl.java:560) at com.google.cloud.datastore.DatastoreImpl.commitMutation(DatastoreImpl.java:553) at com.google.cloud.datastore.DatastoreImpl.delete(DatastoreImpl.java:538) at com.example.datastore.QuickstartSampleIT.deleteTestEntity(QuickstartSampleIT.java:46) at com.example.datastore.QuickstartSampleIT.tearDown(QuickstartSampleIT.java:61) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:364) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:237) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548) Caused by: com.google.datastore.v1.client.DatastoreException: I/O error, code=UNAVAILABLE at com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:136) at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:105) at com.google.datastore.v1.client.Datastore.commit(Datastore.java:87) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc.commit(HttpDatastoreRpc.java:162) ... 39 more Caused by: java.io.IOException: Error getting access token for service account: 400 Bad Request POST https://oauth2.googleapis.com/token {"error":"invalid_grant","error_description":"Invalid JWT Signature."} at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:612) at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:164) at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:149) at com.google.auth.oauth2.ServiceAccountCredentials.getRequestMetadata(ServiceAccountCredentials.java:946) at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:91) at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:159) at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:109) at com.google.cloud.datastore.spi.v1.HttpDatastoreRpc$1.initialize(HttpDatastoreRpc.java:91) at com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:91) ... 41 more Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request POST https://oauth2.googleapis.com/token {"error":"invalid_grant","error_description":"Invalid JWT Signature."} at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1116) at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:609) ... 49 more
A query result from a query that starts at a cursor will say getCursorAfter is actually the default cursor (Cursor{bytes=}
) rather than the cursor it started with.
This creates complications for testing / resuming a result set that was never iterated.
The old Java 8 datastore SDK didn't have this issue.
Start cloud_datastore_emulator on localhost:8081
Run code example
import com.google.cloud.datastore.*;
Datastore ds = DatastoreOptions.newBuilder().setProjectId("project").setHost("localhost:8081").build().getService();
ds.put(Entity.newBuilder(Key.newBuilder("project", "Kind", "name-01").build()).build());
ds.put(Entity.newBuilder(Key.newBuilder("project", "Kind", "name-02").build()).build());
ds.put(Entity.newBuilder(Key.newBuilder("project", "Kind", "name-03").build()).build());
QueryResults<Entity> run = ds.run(Query.newEntityQueryBuilder().setKind("Kind").build());
run.next();
Cursor cursor = run.getCursorAfter();
QueryResults<Entity> run2 = ds.run(Query.newEntityQueryBuilder().setKind("Kind").setStartCursor(cursor).build());
// expected:
assert run2.getCursorAfter().equals(cursor);
// actual: run2.getCursorAfter() is the default cursor until run2.next() is called
The release PR #113 is still in a pending state after several hours
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.