Comments (8)
Hi @stephenh,
Is this tool works for large amount of files?
Thanks and Regards,
Ramesh AR
from mirror.
I'm currently using mirror with ~500,000 files, which are all source code files, i.e. not large video/binary/etc. files. It works really well for that setup.
How many files are you trying to sync?
It looks like 179997852647
nanos is ~exactly three minutes, so there is probably an RPC timeout that could be adjusted.
Ah, there it is:
mirror/src/main/java/mirror/Utils.java
Line 28 in 9e72dff
You could try bumping that up to, say, 10 minutes and seeing what happens/if that fixes it for you.
from mirror.
Hi @stephenh,
Please let me know the way to update the timeout value to 10 min. Is there any way to pass the timeout value from the command line? or Do I need to rebuild the code with 10min?
Thanks and Regards,
Ramesh AR
from mirror.
Yes, right now you'd have to rebuild the code with 10 minutes. I can work on adding a command line flag but would take a few days to get around to.
from mirror.
Thank you @stephenh for your quick help.
from mirror.
Hi @stephenh,
Thank you for the update.
I have updated the timeout value to 10 min and ran the build. The build is running for the past 5 hours. Is it expected?
`<=========----> 76% EXECUTING [5h 18m 49s]
:test > Executing test mirror.IntegrationTest`
Thanks and Regards,
Ramesh AR
from mirror.
No, the tests should take ~2-3 minutes. IntegrationTest
does take longer b/c it uses the real local file system and real client/server instances, so it needs to do some sleep
s etc while doing assertions, so it is not "all in-memory unit test" fast, but definitely not 5 hours.
I've never seen that happen so no good guesses about why, other than try it a few times.
from mirror.
Hi @stephenh,
Thank you for the update.
I have given the "./gradlew clean check shadowJar" to build the jar. It seems the check option is taking more time for me. Later I have give "./gradlew clean shadowJar" and the build completed in mins.
The tool is working fine after I increased the timeout value.
Thank you again for your help.
Thanks and Regards,
Ramesh AR
from mirror.
Related Issues (20)
- Do not respect .gitignore content HOT 7
- "OutOfMemoryError: Java heap space" error while running mirror HOT 4
- Support very large files HOT 1
- Deleting a directory hierarchy leaves empty directory
- ignoring directories with many files HOT 2
- About package name HOT 3
- Can we use the diff port instead of '49172'? HOT 5
- How to keep the file permissions? HOT 1
- CentOS docker image HOT 2
- Question: What causes lag between watchman and mirror sending a file? HOT 4
- java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.facebook.watchman.WatchmanException: unknown command watch-project HOT 1
- logs are not rolling. HOT 2
- exclude file pattern issue HOT 3
- Issue with bulk file uploads HOT 1
- restricting server to only one interface HOT 7
- Output per-file status info to systemd/syslog HOT 1
- Specify log file path? HOT 3
- Mirroring directories on same server (Docker) HOT 1
- How do I debug mirror? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mirror.