Git Product home page Git Product logo

file-management-system's Introduction

File Management System

File Management System is a hybrid file storing system. The file information is saved into database. The file content is located into logical disk. The system is exposed via API.

Summary

  1. Functionalities
    • 1.1 File Upload
    • 1.2 File Download
    • 1.3 Storage mode
  2. Getting Started (Prerequisites, Installing)
    • 2.1 Prerequisites
    • 2.2 Database installation
      • 2.2.1 H2
      • 2.2.2 MySQL
      • 2.2.3 PostgreSQL
  3. Running the tests
  4. Deployment 4.1 Deployment & run on LINUX environment 4.2 Deployment & run on WINDOWS environment
  5. Swagger usage
  6. Built With
  7. Contributing
  8. Versioning
  9. Authors
  10. License
  11. Donation

1. Functionalities

1.1 File Upload

1.2 File Download

1.3 Storage mode

2. Getting Started

Clone or download a copy of this project.

2.1 Prerequisites

This project requires Java 1.8, Maven and at least one database (PostgreS, H2, MySql).

2.2 Database installation

2.2.1 H2

No installation is required. The spring.datasource.url is the one required property which should be set. By default, the username is sa with empty password. Two modes: in memory and file storage. See the application.properties file for more details related configuration. To access default H2 Console use this link: http://localhost:8081/h2-console/ , where 8081 is the server port.

2.2.2 MySQL

CREATE DATABSE fms;

Note: in case that you run the application starting with MySQL 8.0.4, please execute the following query:

ALTER USER '${USER}'@'localhost' IDENTIFIED WITH mysql_native_password BY '${PASSWORD}';
-- where ${USER} and ${PASSWORD} should be provided. 

2.2.3 Postgres

Install PostgreSQL. It is required to create a database:

Please, run the following commands if it is the case:

createuser -U postgres -s Progress

Please, run the following command to import a database (if it is the case):

pg_restore -d DATABASE_NAME <  PATH/BACKUP_FILE_NAME.sql

To create the JAR file please use the following command:

mvn clean package

3. Running the tests

All available unit / integration tests are in package: src/test/java. The main rule is: one unit test class for each java class.

4. Deployment

If the build (the jar file) is ready then the application can be run. Please, use the following command to run the application:

XXX:file-management-system xxx java -jar target/file-management-system-X.Y.Z-SNAPSHOT.jar

4.1 Deployment & run on LINUX environment

In case if application is run in a linux based instance, please create the following folders:

  1. /app - folder where the JAR is located;
  2. /app/log - folder which will contains the logs;
  3. /app/config - folder which will contains the application configuration files;
  4. /app/file-db - production or working folder where service will save physically the content;
  5. /app/test-db - test folder where service will save physically the content;

To run the application we have the following options:

nohub java -jar file-management-system-1.0.7.jar --spring.config.location=/app/config/application.properties &

To show last NUM_OF_RECORDS from a FILE_NAME linux command:

tail -n NUM_OF_RECORDS FILE_NAME

4.2 Deployment & run on WINDOWS environment

The application was developed on Windows environment. We did not test how the application works on PRODUCTION on WINDOWS environment.

To run application, please run this command:

java -jar /projects/app/file-management-system-1.0.7.jar --spring.config.location=/projects/app/config/application.properties

5. Swagger usage

You can use swagger for testing proposes:

URL: APP_HOST/swagger-ui.html  . For example:  http://localhost:8081/swagger-ui.html

6. Built With

  • Java - Java technology allows you to work and play in a secure computing environment. Java allows you to play online games, chat with people around the world, calculate your mortgage interest, and view images in 3D, just to name a few.
  • Spring Security - Spring Security is a powerful and highly customizable authentication and access-control framework. It is the de-facto standard for securing Spring-based applications.
  • Spring Boot - Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can "just run".
  • Spring Data - Spring Data’s mission is to provide a familiar and consistent, Spring-based programming model for data access while still retaining the special traits of the underlying data store.
  • Spring Data JPA - Spring Data JPA, part of the larger Spring Data family, makes it easy to easily implement JPA based repositories. This module deals with enhanced support for JPA based data access layers. It makes it easier to build Spring-powered applications that use data access technologies.
  • Spring Doc - Java library helps to automate the generation of API documentation using spring boot projects..
  • H2 - H2 is a relational database management system written in Java. It can be embedded in Java applications or run in client-server mode.
  • PostgreSQL - PostgreSQL, also known as Postgres, is a free and open-source relational database management system (RDBMS) emphasizing extensibility and technical standards compliance. It is designed to handle a range of workloads, from single machines to data warehouses or Web services with many concurrent users. It is the default database for macOS Server, and is also available for Linux, FreeBSD, OpenBSD, and Windows.
  • Maven - Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information.

7. Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

8. Versioning

We use SemVer for versioning.

9. Authors

Sergiu Drahnea is the author and the person who developed the system. If you wish to contact then please drop a message to my LinkedIn account: https://www.linkedin.com/in/sergiu-drahnea/

10. License

MIT License in general. If you want to run the system in production and have questions or need advices then please drop a message.

11. Donation

  • PayPal - any donation is welcomed in case that you was pleased with this work :p
  • EGLD - Address: erd1t3t5m8v7862asdh48nq820shsmlmuw9jpm87qw25cvch7djpkapskgq4es
  • TROY - Address: bnb136ns6lfw4zs5hg4n85vdthaad7hq5m4gtkgf23 and Memo: 100079140
  • PHB - Address: bnb136ns6lfw4zs5hg4n85vdthaad7hq5m4gtkgf23 and Memo: 100079140
  • HOT - Address: 0x1ebfc62e2510f0a0558568223d1d101d0cf074b2
  • VET - Address: 0x1ebfc62e2510f0a0558568223d1d101d0cf074b2
  • TRX - Address: TRe8xSkGqpS73Nhk6bnvW34aiJoRTmZs8N
  • BTT - Address: TRe8xSkGqpS73Nhk6bnvW34aiJoRTmZs8N

file-management-system's People

Contributors

sdrahnea avatar

Stargazers

jongwon avatar  avatar Jose Marin avatar

Watchers

James Cloos avatar  avatar

file-management-system's Issues

[FMS-13] Add java client for fms

Add java client for fms as a project, so anybody should be able to see an example about show to exchange the dates between a supposed client application and file management system.

Should an sample for each end-point.

[FMS-6] Implement storage strategy: FILE

Storage strategy - FILE (no storage into database)

File storage based on the following rule: ${file.db.location} / ${tenant} / ${file_id}

To storage the file, the request will contains: tenant. To return the file_id.
To receive the file, the request will contains: tenant and file_id.

[FMS-7] Implement storage strategy: FILE_PER_DATE

Storage strategy - FILE_PER_DATE (no storage into database)

File storage based on the following rule: ${file.db.location} / ${tenant} / ${date}/ ${file_id}

To storage the file, the request will contains: tenant. To return the file_id and date (when file was created).
To receive the file, the request will contains: tenant / date / file_id.

[FMS-8] Implement storage strategy - FILE_PER_YEAR_DATE

Storage strategy - FILE_PER_YEAR_DATE (no storage into database)

File storage based on the following rule: ${file.db.location} / ${tenant} / ${year}/ ${date} /${file_id}

To storage the file, the request will contains: tenant. To return the file_id and date (when file was created).
To receive the file, the request will contains: tenant / year / date / file_id.

[FMS-1] Unable to process parts as no multi-part configuration has been provided

java 9 / 11

2021-05-04T15:57:30,661 ERROR [http-nio-8081-exec-10] o.a.j.l.DirectJDKLog: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.web.multipart.MultipartException: Failed to parse multipart servlet request; nested exception is java.lang.IllegalStateException: Unable to process parts as no multi-part configuration has been provided] with root cause
java.lang.IllegalStateException: Unable to process parts as no multi-part configuration has been provided
at org.apache.catalina.connector.Request.parseParts(Request.java:2856)
at org.apache.catalina.connector.Request.getParts(Request.java:2824)
at org.apache.catalina.connector.RequestFacade.getParts(RequestFacade.java:1098)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:95)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.(StandardMultipartHttpServletRequest.java:88)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.(StandardMultipartHttpServletRequest.java:72)
at org.springframework.web.multipart.support.MultipartResolutionDelegate.resolveMultipartArgument(MultipartResolutionDelegate.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestPartMethodArgumentResolver.resolveArgument(RequestPartMethodArgumentResolver.java:128)
at org.springframework.web.method.support.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:121)
at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:170)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:894)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:652)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1707)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:836)

Process finished with exit code -1

[FMS-10] Implement storage strategy - FILE_PER_YEAR_MONTH_DAY

Storage strategy - FILE_PER_YEAR_MONTH_DAY (no storage into database)

File storage based on the following rule: ${file.db.location} / ${tenant} / ${year}/ ${month}/ ${day} /${file_id}

To storage the file, the request will contains: tenant. To return the file_id and date (when file was created).
To receive the file, the request will contains: tenant / year / month / day / file_id.

[FMS-9] Implement storage strategy - FILE_PER_YEAR_MONTH_DATE

Storage strategy - FILE_PER_YEAR_MONTH_DATE (no storage into database)

File storage based on the following rule: ${file.db.location} / ${tenant} / ${year}/ ${month}/ ${date} /${file_id}

To storage the file, the request will contains: tenant. To return the file_id and date (when file was created).
To receive the file, the request will contains: tenant / year / month / date / file_id.

[FMS-14] Delete data older by age

Implement functionality which allow application to remove the file by age.

Add new property: age in days / months / years
Example:
file.cleanup.age=100
file.cleanup.age.type=DAY or MONTH. or YEAR

File cleanup is not enable:

  • by default, no property was declared
  • properties are declared, but empty

Should be like a scheduler which run at least once per day and execute file clean up.

Note: if the application uses the database then the records should be disabled from the all possible business flows.

[FMS-3] add multi-tenant possibility

the application should be storage files based on tenant.
The default file structure should be:
{$file.db.location} / {$tenant} / {$DEFAULT_STRATEGY_DIRECTORY: current_date} / {$FILE_ID}

where properties (except the last one) are folders and if not present then create :

  1. file.db.location - base root where will be saved all files content
  2. tenant - folder for related tenant
  3. DEFAULT_STRATEGY_DIRECTORY - in future to be able to create folders based on different strategies. Currently it should be create a folder based on current date
  4. FILE_ID - the id which is received from the client

[FMS-11] FILE_PER_YEAR_MONTH

Storage strategy - FILE_PER_YEAR_MONTH_DAY (no storage into database)

File storage based on the following rule: ${file.db.location} / ${tenant} / ${year}/ ${month} /${file_id}

To storage the file, the request will contains: tenant. To return the file_id and date (when file was created).
To receive the file, the request will contains: tenant / year / month / file_id.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.