Git Product home page Git Product logo

dscpullserveradmin's Introduction

DSCPullServerAdmin

master dev psgallery
Build status Build status PowerShell Gallery
codecov codecov

A module to work with the DSC PullServer EDB (ESENT), MDB (Access) and SQL databases and provide a better Admin oriented Pull Server experience.

Update 12/11/2018: As of version 0.4.0, the module is able to handle MDB Databases! (For now this requires Windows PowerShell with an additional installation of Microsoft Access Database Engine 2016 Redistributable)

Update 11/20/2018: As of version 0.3.0, the module is able to handle the EDB with all features again. Get/Set/Remove/New is (in experimental) support!

The module enables you to move data from the EDB or MDB database to a SQL database. This allows you to "upgrade" your Pull Server implementation to make use of the SQL backend introduced in Windows Server RS4 (1803) and Server 2019. It is also capable of doing migrations from any source database type to any destination database type as of version 0.3.0!

The module supports accessing, manipulating, removing Devices (Legacy LCMv1, ConfigurationId), Registrations (LCMv2, AgentId) and StatusReports (LCMv2). All this data can be easily moved over to SQL as well.

Note that the current state of this module is experimental. Use at your own risk and if possible always create a backup!

Thanks!!

A big thank you goes out to @rdbartram and @gaelcolas who both brought in some major contributions. Thank you both!

Examples

EDB

Connect with EDB (the database cannot be in use by another process!) and get Devices and Registrations out of it.

edb01

Get StatusReports out of the EDB.

edb02

edb03

SQL

Connect with SQL and Get, Set, Create and Remove data.

sql01

sql02

sql03

Copy Data EDB to SQL

All data can be copied over.

Note, the function has been renamed to Copy-DSCPullServerAdminData as it is now capable of handling any source database type to any destination database type!

edb2sql01

edb2sql02

Class diagram

This project is PowerShell class based. Find here the Class diagram structure:

Diagram

dscpullserveradmin's People

Contributors

bgelens avatar gaelcolas avatar rdbartram avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dscpullserveradmin's Issues

StatusReport Top should move backwards

There is no automatic id assignment per record so it is not possible to sort by id and select top from desc.

A workaround is needed as the Top to be expected are last inserted records.

For EDB this means using TryMoveLast and TryMovePrevious

For SQL and MDB another workaround is needed.

move data in edb does not make it available for the pull server

When moving data from mdb to edb or sql to edb, the pull server will not see the inserted records.

moving data out of edb to mdb or sql will make it available to the pull server.

Investigate what needs to be done before the pull server will see the records. One thing I see is tat inserts by the pull server code make use of update.SaveAndGotoBookmark(); which is not implemented in the current dscpullserveradmin code for set-eserecord

Default EDB File Path parameter

Hi,
it would be nice if Paramter of ESEPath has a default value like
"C:\Program Files\WindowsPowerShell\DscService\Devices.edb"

Thanks

New-DSCPullServerAdminConnection error when passing credential

When I try to pass a credential to New-DSCPullServerAdminConnection, I get the following error:

PS C:> New-DSCPullServerAdminConnection -SQLServer SERVERNAME-Database DSC -Credential (Get-Credential)

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential
Invoke-DSCPullServerSQLCommand : Exception calling "Open" with "0" argument(s): "Login failed for user 'CORP\mcelreathm'."
At C:\Users\mcelreathm\Documents\WindowsPowerShell\Modules\DSCPullServerAdmin\0.1.5\DSCPullServerAdmin.psm1:1081 char:19

  • ... estResult = Invoke-DSCPullServerSQLCommand -Connection $testConnectio ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : NotSpecified: (:) [Write-Error], MethodInvocationException
    • FullyQualifiedErrorId : SqlException,Invoke-DSCPullServerSQLCommand

I'm using an account that is a sysadmin on the SQL server which is also the account I'm logged in as. If I omit the -Credential parameter, it works fine.

PS C:> $PSVersionTable

Name Value


PSVersion 5.1.15063.1292
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.15063.1292
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1

Filtering options Get-DSCPullServerAdminStatusReport

Add filtering options to Get-DSCPullServerAdminStatusReport like:

  • Newest (default to 1)
  • Oldest (default to 1)
  • Status (success, failed, ??)
  • OperationType (LocalConfigurationManager, Consistency, Initial, ??)
  • ...

Instance Name already in use

Running
Get-DSCPullServerAdminReport -ESEPath 'C:\Program Files\WindowsPowerShell\DscService\Devices.edb' | select -Last 1
once works. When i run the same code a second time i get the following message:
`Get-DSCPullServerAdminReport : Instance Name already in use
At line:1 char:1

  • Get-DSCPullServerAdminReport -ESEPath 'C:\Program Files\WindowsPowerS ...
  •   + CategoryInfo          : NotSpecified: (:) [Get-DSCPullServerAdminReport], EsentInstanceNameInUseException
      + FullyQualifiedErrorId : Microsoft.Isam.Esent.Interop.EsentInstanceNameInUseException,DSCPullServerAdmin.src.CmdLets.GetDSCPullServerAdminReport`
    
    

This only happens if you start the cmdlet and break it while it is running (CTRL+C)

Env: Windows Server 2016
PS: 5.1.14393.693

Database cleanup

Hi,
first thanks for this great project.
Since the database Devices.edb is growing and growing it would be nice if we could clean up StatusReports table. A switch how much days back we would like to have would be nice.

Thanks

xPlat SQL Support

The Module already is capable of running xPlat when SQL databases are used (MDB and EDB are not supported on non-Windows).

However, the use of RequiredAssemblies for Microsoft.Isam.Esent.Interop in the module manifest makes it impossible to import on Linux / Mac.

To fix this, the module needs another way of importing this assembly (Windows only) and make it clear to a non-Windows user that MDB and EDB are not available

Get-DSCPullServerAdminStatusReport throws "Cannot open any more tables" error

I know I mentioned this on Slack but I thought it would probably be best to raise it here instead.

I am using version 0.4.2 on a Server 2012 R2 pull server and have stopped IIS with iisreset /stop before making a copy of the Devices.edb file to a temporary location. Then against this copy of the DB I run the following commands and get the error as shown below.

Import-Module DSCPullServerAdmin
New-DSCPullServerAdminConnection -ESEFilePath '.\TempDevices.edb'
Get-DSCPullServerAdminStatusReport

Get-DSCPullServerESERecord : Cannot open any more tables (cleanup already attempted)
At line:2092 char:17
+                 Get-DSCPullServerESERecord @eseParams 
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (:) [Write-Error], EsentTooManyOpenTablesException
+ FullyQualifiedErrorId : Microsoft.Isam.Esent.Interop.EsentTooManyOpenTablesException,Get-DSCPullServerESERecord 

Have you any idea why I am running into this?

As I mentioned today on Slack I had no issues when using the Copy-DSCPullServerAdminData command to create a fresh DB and import all my node registrations across. This temp DB is a copy of the fresh DB so there shouldn't be any old StatusReports hanging around only ones sent since this morning.

Cheers.

MDB support

To introduce MDB support we need access to System.Data.OleDb.OleDbConnection which is afaik only available in 64 bit when Microsoft Access Database Engine is installed. When this is installed, PowerShell 5.1 is needed to access it.

  • First create MDB support
  • Research if PowerShell 6 can be supported (the rest of the module work with 6 on Windows). UPDATE Test on type available. If type is not available write error suggesting to use PSv5.1. If PSv6.x based on newer netcore does have the type in the future, the Error won't be written.
  • Research if a library can be shipped as part of the module to prevent additional installation requirements. UPDATE: Won't do this. Instead write error with download requirements
  • Like behaves different on OleDb. It requires a wildcard to be appended in some cases or it won't work. In other cases, it just doesn't work. Research how to solve this. (Note, behavior is Server Core, Windows 10 works as expected). UPDATE: Decided to exclude wildcard support for MDB connections
  • Add tests

Provide ability to move EDB content to SQL DB

Now we have SQL support in Server 1803 (SAC) / 2019 (LTSC), it would be nice if we could get the content out of the EDB and insert it directly in the SQL Database to preserve existing node registrations and reporting data.

Right now my thinking is to have the ability to either create a new SQL DB from scratch or be able to connect up with an existing one.

  • Cmdlet/Function to create a new DB
  • Cmdlet/Function to connect with existing DB (created by either the Pull Server itself or by using the previously mentioned Cmdlet/Function)
  • Cmdlet/Function to convert EDB data to SQL INSERT Statements and execute on DB if not exists

Statusdata in statusreport can have invalid json

This can cause errors which are terminating when happening in ese mode

  • Investigate why this happens
  • Handle these errors more gracefully
  • Determine if it is caused by moving data into edb by SerializeObjectToColumn (maybe needs additional ConvertTo-Json before Serialize will happen, unsure)

GitHub release

When releasing to master, include the GitHub release (tag with version same as published on the gallery)

Right now, release in GitHub is manual and will trigger a new build, need to filter this as it will publish a new version to the gallery.

Revisit unit tests to make use of testcases

Currently there are a lot of tests that are basically the same but just targeting a different database provider.
This results in a lot of duplication in the unit test files that could potentially be resolved by moving stuff to testcases

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.