Comments (4)
Instances of session
in transfer.py
:
$ cat transfers/transfer.py | ack -A 2 -B 2 session
def get_status(am_url, am_user, am_api_key, ss_url, ss_user, ss_api_key,
unit_uuid, unit_type, session, hide_on_complete=False,
delete_on_complete=False):
"""
--
unit_uuid, unit_info.get('sip_uuid'))
# Update DB to refer to this one
db_unit = session.query(models.Unit).filter_by(
unit_type=unit_type, uuid=unit_uuid).one()
db_unit.unit_type = 'ingest'
--
def call_start_transfer_endpoint(
am_url, am_user, am_api_key, target, transfer_type, accession,
ts_location_uuid, session):
"""Make the call to the start_transfer endpoint and return the unapproved
directory name.
--
new_transfer = models.Unit(
path=target, unit_type='transfer', status='FAILED', current=False)
session.add(new_transfer)
return None
--
def start_transfer(ss_url, ss_user, ss_api_key, ts_location_uuid, ts_path,
depth, am_url, am_user, am_api_key, transfer_type,
see_files, session, config_file):
"""
Starts a new transfer.
--
:param bool see_files: If true, start transfers from files as well as
directories
:param session: SQLAlchemy session with the DB
:returns: Tuple of Transfer information about the new transfer or None on
error.
"""
# Start new transfer
completed = {x[0] for x in session.query(models.Unit.path).all()}
target = get_next_transfer(
ss_url, ss_user, ss_api_key, ts_location_uuid, ts_path, depth,
--
am_url=am_url, am_user=am_user, am_api_key=am_api_key, target=target,
transfer_type=transfer_type, accession=accession,
ts_location_uuid=ts_location_uuid, session=session)
if not target_name:
LOGGER.info("Cannot begin transfer with target_name: %s", target_name)
--
uuid=result, path=target, unit_type='transfer', current=True)
LOGGER.info('New transfer: %s', new_transfer)
session.add(new_transfer)
break
LOGGER.info('Failed approve, try %s of %s', i + 1, retry_count)
--
new_transfer = models.Unit(
uuid=None, path=target, unit_type='transfer', current=False)
session.add(new_transfer)
return None
--
os.path.join(THIS_DIR, 'transfers.db')))
session = models.Session()
# Check for evidence that this is already running
--
current_unit = None
try:
current_unit = session.query(models.Unit).filter_by(current=True).one()
unit_uuid = current_unit.uuid
unit_type = current_unit.unit_type
--
# Get status
status_info = get_status(am_url, am_user, am_api_key, ss_url, ss_user,
ss_api_key, unit_uuid, unit_type, session,
hide_on_complete, delete_on_complete)
LOGGER.info('Status info: %s', status_info)
--
if status == 'PROCESSING':
LOGGER.info('Current transfer still processing, nothing to do.')
session.commit()
return 0
# If waiting on input, send email, exit
--
)
current_unit.microservice = microservice
session.commit()
return 0
# If failed, rejected, completed etc, start new transfer
--
ss_url, ss_user, ss_api_key, ts_uuid, ts_path,
depth, am_url, am_user, am_api_key,
transfer_type, see_files, session, config_file)
session.commit()
return 0 if new_transfer else 1
from automation-tools.
Some best practice about sessions and closing them can be found here on the Sqlalchemy site: https://docs.sqlalchemy.org/en/latest/orm/session_basics.html#when-do-i-construct-a-session-when-do-i-commit-it-and-when-do-i-close-it
tl;dr;
As a general rule, keep the lifecycle of the session separate and external from functions and objects that access and/or manipulate database data. This will greatly help with achieving a predictable and consistent transactional scope.
Make sure you have a clear notion of where transactions begin and end, and keep transactions short, meaning, they end at the series of a sequence of operations, instead of being held open indefinitely.
from automation-tools.
Resolved via: bc93ff2
from automation-tools.
NOICE!
from automation-tools.
Related Issues (20)
- Problem: processing hangs because UUID for ingest is missing in the Sqlite database HOT 1
- Problem: installation instructions HOT 1
- Potential dependency conflicts between automation-tools and urllib3 HOT 1
- Delete on complete doesn't work in transfer.py HOT 1
- create_dip.py script changes file names from BagIt transfers HOT 1
- Problem: Latest AT fails in AM<=1.14 because python<3.8 HOT 1
- Problem: fits:fslastmodified is not always available as a source for setting the modified date of DIP files HOT 5
- Problem: AMClient instance attributes are used for request-specific actions
- Problem: Transfers.py uses 'r+' fopen mode but probably only requires 'w'
- Problem: Make the Automation Tools take advantage of Python Setup Tools
- Problem: There are a larger number of filetypes than we can lookup in dataverse pre-transfer HOT 1
- Problem: Add delete on completion pull-request HOT 1
- Problem: Sample README.md shell scripts are all on one line HOT 1
- Problem: Need a script to enable bulk re-ingest
- Problem: Apply ambv/black across AM codebase HOT 2
- Problem: It might be more flexible to autogenerate portions of README.md from amclient.py HOT 1
- Problem: Consider ways of removing repetition when calling amclient.py HOT 1
- Problem: no issue template HOT 1
- Problem: Consider making the installation instructions a little more clear HOT 1
- Problem: this repo has no contributing.md HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from automation-tools.