Comments (15)
@mslinn Thanks a lot for your improvement suggestions. They completely make sense to me. I'd highly appreciate if you could make a pull request with your proposed changes and to replace UserGuideDoc.scala
with the those developed in your repo and mention them in the user guide.
Updated to Scala 2.12.1 and SBT 0.13.13.
We still need to default to Scala 2.11 because of the dependencies in the Spark Adapter. http://rbmhtechnology.github.io/eventuate/adapters/spark.html. Although we're additionally releasing 2.12 versions of Eventuate without the Spark adapter we can only default to Scala 2.12 when all Spark dependencies support that as well.
In doing the above, I discovered that the mysterious LevelDB mentioned in the docs was actually a Scala port of the Google LevelDB project, originally written in C++.
We're actually using the C++ version of LevelDB with a JNI wrapper and not a Java port of LevelDB.
from eventuate.
Happy to make a PR when ready. I have a lot to learn!
- I have copied the Sphinx
user-guide.rst
into my project. I have not used Sphinx before but my intention is to make improvements to the source code and the docs together, then later put the files back to the original project, hopefully as an SBT subproject. - What is EventsourcedActorsUpdated.scala for? It is not referenced by anything anywhere.
- What is EventCommunication for? It is not referenced by anything anywhere.
- I created application.conf to configure Akka, but encountered the need for a serializer in order to suppress a message: "Using the default Java serializer for class [doc.EventsourcedActors$Appended] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'". I found https://rbmhtechnology.github.io/eventuate/reference/event-sourcing.html#custom-serialization but would appreciate someone taking care of that for me with a PR against my git project.
from eventuate.
from eventuate.
... then later put the files back to the original project, hopefully as an SBT subproject.
Making the Sphinx code examples a separate sbt subproject probably won't work as this directory is already an unmanaged source in the settings of the root sbt project. At the moment, Eventuate documentation references code examples at three different kinds of locations:
- in
src/sphinx/code
. Contains examples that only havecore
,logLeveldb
andlogCassandra
as dependencies. - in
eventuate-example*
: More examples with more dependencies than insrc/sphinx/code
. - in
src/main
of individual modules. References to Eventuate main code directly.
We could either stay with that pattern or decide to move all examples from src/sphinx/code
to eventuate-examples
. This could make it easier to execute the examples as eventuate-examples
is a separate sbt subproject.
EventsourcedActorsUpdated
...EventCommunication
... It is not referenced by anything anywhere.
It is referenced by .rst
files to be included as code snippets. Main purpose of the code in src/sphinx/code
is to have example code that compiles i.e. if there's a compile error in one of the examples, the documentation build (i.e. sbt makeSite
) fails too.
... need for a serializer ...
I think it's completely ok to use Java serialization for the examples together with a akka.actor.warn-about-java-serializer-usage = false
configuration setting.
I've searched and experimented to produce a formatted link using Sphinx, but failed.
Me too, didn't find a way to generate formatted links. We therefore followed the convention that links are always unformatted.
from eventuate.
-
I found that the stylesheets were peculiar and suboptimal, so I made the minimum changes necessary for them to be easier to read without destroying the current look and feel. These changes will affect every page of the documentation site.
-
I now understand why the user guide source needs to be bundled alongside the rest of Eventuate. However, the source should not be buried deep within. Lets put all the example projects under one top-level directory, called
examples
, and put any old code examples underexamples/old
so people know what to expect. If/when the old code examples get updated they would be graduated up a level, for example fromexamples/old/interactive-confict-resolution
toexamples/interactive-confict-resolution
. If any other documents currently undercode
(EventLogDoc.scala
,EventRoutingDoc.scala
,EventSourcingDoc.scala
, andReliableDeliveryDoc.scala
) also contain source code, those should eventually be migrated to theexamples/
directory (note I have not volunteered for this task, separate issues would need to be created for each document.)
eventuate/
├── examples
│ ├── adapter
│ │ ├── spark
│ │ ├── stream
│ │ └── vertx
│ ├── old
│ │ └── interactive-confict-resolution
│ └── user-guide
└── other
Notice that the directory names are short, and form a descriptive relative path name that does not contain redundant use of the word eventuate
. For example, examples
instead of eventuate-examples
, and examples/adapter/spark
instead of eventuate-examples/eventuate-examples-spark
.
-
SBT's multiproject support improved with v0.13.13. The example programs should each be standalone SBT projects, with top-level
.sbt
files using SBT 0.13.13+. This means users will easily be able to understand the minimal requirements to make each program work, instead of the current undocumented/magical/complex multi-project setup. -
Not only should the code examples be easy to find and their usage and status be intuitive, they should work without odd errors or output that diminishes a potential user's confidence.
a. The serializer message falls into this category. It should be addressed. There is only one chance for a first impression.
b. It is clear that none of the User Guide's Java programs have ever been runnable from a command line. If Java support is a priority then this should be addressed, otherwise Java support should be labeled as being a second priority, and one in need of volunteers.
c. Most of the Scala programs do not do anything or produce output that helps a user understand the points made in the document. The exception is the first program (
ActorExample.scala
). I'm not going to take this task on – someone who knows the project could probably do this fairly quickly. I would be happy to discuss the reader's needs from the point of view of the User Guide with that person.
This last point needs to be done before I can complete my task of updating the user guide, so I have made the directory changes described above and merged my work to date into a new branch (newDocs
) of my fork of Eventuate. I am now awaiting a volunteer to polish the Scala code examples. If that person, or another volunteer, then moves on to polish the Java code examples, and makes the necessary changes so the Java programs match the corresponding Scala program in terms of functionality, I will rewrite the User Guide to suit.
Update Mar 20/17 11:24am PT: Because I have made changes to the Scala and Java programs, the person/people who assume the task of updating them should extend my work; as of right now the most current version of the sample programs is found in my Eventuate fork in the newDocs
branch.
from eventuate.
@mslinn thanks for all the effort you put into it so far. Looks good to me at a first glance and would like to make a more detailed review in several smaller pull requests instead of a single large one. For example PRs could cover:
- restructure examples to have only a single
examples
root and update the references in the documentation accordingly. - improve the user guide as you did already in your fork.
- extend the examples to make them self-contained and runnable with all the improvements you suggested. Maybe these can be separate PRs for Scala and Java examples. I cannot promise that I or another volunteer will have the time in the near future to work on that as I'm working on other high-priority tasks at the moment.
- further improve to the user guide based on what was done in the previous PR.
- stylesheet enhancements. Here we need to make sure that the changes are still compatible with RBMH requirements and style guidelines. I will have to check that.
- ...
This procedure will allow us to make progress in several smaller steps and easier for others to review and discuss changes. Regarding the code examples and the separate sbt root: we need ensure that the examples still compile when generating the documentation. Not sure if this is possible if the documentation and the examples use different sbt roots (but I'm not an sbt wizard and that's only a minor issue).
from eventuate.
Makes perfect sense.
- I already updated all references throughout the entire project, including the references in the documentation. My fork has Travis set up to work with the new branch I created and it passes. The badge is shown here and this means the root project still compiles.
- You can check the style sheet changes by cloning my fork, switching to the
newDocs
branch, and running the following. Viewtarget/html/user-guide.html
in a web browser.
sphinx-build -b html src/sphinx/ target/html/ &> sphinx.log
- To begin with I could make a PR with just the two stylesheets I modified. A small thing, easy to deal with.
from eventuate.
To begin with I could make a PR with just the two stylesheets I modified. A small thing, easy to deal with.
Please do that, thanks!
from eventuate.
@mslinn please sign the CLA when making your first PR.
from eventuate.
I made Akka serializers and deserializers for the Scala versions of VectorTime and Versioned, and wrote tests. This code probably belongs in the main project, since everyone probably would benefit from a simple way to serialize/deserialize these objects. If the code was ported to the main project, the implicit classes would not be required.
I also updated the User Guide code in my fork to take advantage of these serializers / deserializers, plus wrote a few more to handle Int
and Long
.
I did not make Java serializers or deserializers for VectorTime
or Versioned
. It is not hard to do, but I get the sense it is not important so I did not bother. However, application.conf
is already set up, just need to uncomment the entries if and when they are implemented.
from eventuate.
eventuate-core
already contains a protobuf serializers for VectorTime
and Versioned
(protobuf definition). In general, all Eventuate objects that are persisted to an event log or sent over the network have a corresponding protobuf serializer (see also custom serialization).
I'm not sure if it is helpful to have custom serializers for all the messages/events used in the examples (as it bloats them significantly). As a user I'd find a single/separate example, how to write and configure a custom serializer more helpful, especially if these are protobuf or Avro serializers, as they are needed for supporting event schema evolution.
from eventuate.
Thanks for pointing me to the serialization documentation. I agree that the User Guide code bloats when serialization is added (seems to be at least 2x larger).
I did the following on my newSerializeDocs
branch:
- Added a comment to the User Guide that states "The above serialization warning is addressed in the Custom Serialization document."
- Moved some (not all) of the serialization code that I wrote yesterday to a new SBT project in
examples/serialization
, and deleted the rest of the serialization code from the User Guide.
When ready, I'll make a new PR for the changes to the Serialization document, where we can address any issues that may exist with the code I wrote, and discuss best practices. For example, I used org.apache.commons.lang3.SerializationUtils for serializing objects that can be cast to Serializable
, and java.nio.ByteBuffer
for serializing Int
and Long
. If this is inadvisable, a few sentences about why this is so should be added, and guidance provided.
BTW, I could not find any reference to Apache Avro in the Eventuate code base. Presumably Avro is merely one serialization option, in addition to Google Protocol Buffers / ScalaPB, Apache Thrift / Scrooge, Pickling, Kryo / Chill, SerializationUtils
, and java.nio.ByteBuffer
.
BTW, In a performance benchmark, ScalaPB provided the overall fastest results, which is nice, because it is merely a Scala wrapper around Google Protocol Buffers.
from eventuate.
Sounds good.
where we can address any issues that may exist with the code I wrote, and discuss best practices
Let's only add an example how to write a custom protobuf serializer and remove the serializers that use SerializationUtils
(as they just do Java serialization) and low-level ByteBuffer
. That's all we need to have a best practices example.
I could not find any reference to Avro in the Eventuate code base
Eventuate uses protobuf only. Applications can use whatever they want.
from eventuate.
I updated my newSerializeDocs branch from master and pushed to my eventuate
fork. There are lots of changes. I don't know how the restructuring could be done in smaller pieces without doubling the work.
- Code examples are all standalone SBT projects that compile, and are located under
examples
. - All of the various documents have been edited to point to the new location.
- Some of the code examples were quite long, so I broke them into smaller chunks and updated the markup.
- The build passes
- I wrote comments with FIXME and README so you can find them.
from eventuate.
@mslinn instead of pointing to changes in other repos, please make one or more PRs and let's discuss things there. Thanks for your understanding
from eventuate.
Related Issues (20)
- Emit progress data with processed messages
- Allow EventsourcedProcessor to emit events with custom aggregation IDs HOT 12
- Events created by EventsourcedProcessor lack system timestamp
- Common event processing model
- LogEventDispatcherSpec failed
- Improve logging when CassandraEventLog stops itself due to errors when writing
- Add documentation on how to setup akka remoting with TLS
- Support disaster recovery with limited event recovery
- Document that RBMH Eventuate is not Eventuate.io
- Persist on event may persist events again in case of disaster recovery HOT 3
- Drop support for optional persistOnEventId in PersistOnEventRequest
- Upgrade to Akka 2.5
- Publish .proto files
- Potentially Inconsistent EventLogClock snapshot in Cassandra backend in case of disaster recovery
- Is this thing still on? HOT 2
- Adapt replication batch size dynamically HOT 1
- Using EventsourceProcessors in combination with Replication may lead to loss of processed events HOT 1
- Memory consumption of `LeveldbNumericIdentifierStore` HOT 2
- Replay from Cassandra stops prematurely if there are large sequence number gaps covering empty partitions HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eventuate.