Git Product home page Git Product logo

colemanja91 / pyeloqua Goto Github PK

View Code? Open in Web Editor NEW
22.0 9.0 13.0 3.21 MB

NOTICE: I have not worked in Eloqua in years, and many parts of this package are likely out-of-date. Please feel free to fork and continue the work for the community (if you still have to work with Eloqua, you have my condolences). Python wrapper functions for Eloqua APIs

License: Other

Python 100.00%
eloqua-api marketing marketing-automation python

pyeloqua's Introduction

NOTICE - No longer maintained

I have not worked in Eloqua since 2017, and therefore have not maintained this package. I have used alcohol to erase as much of my working memory of Eloqua as possible. If you still have to work with Eloqua, you have my condolences. Please feel free to fork this repo and continue the work (if you do, please consider sharing with the community of other folks who currently suffer the same fate).

pyeloqua

Python wrapper functions for Eloqua APIs, tested with Python 2.7 and Python 3.3 - 3.6.

Documentation is your friend (http://docs.oracle.com/cloud/latest/marketingcs_gs/OMCAC/index.html) - if you can't do it in the API, you can't do it with this module.

NOTICE I'm in the middle of a rebuild that will result in breaking changes to existing uses. This is to facilitate better unit testing and meet coding standards and best practices. Right now, if you upgrade to the latest version (release v0.4.0), you will receive warnings when attempting to use the old methods. After a few minor version releases, when the new code is stabilized, the deprecated code will be removed entirely. Please continue using release v0.3.5 until you have updated your code.

Please feel free to let me know of any problems by filing an issue on Github.

What can the API do? The Eloqua APIs are for the import and export of data from an existing Eloqua instance.

Examples

##Getting started

You need an Eloqua user account with at least Advanced Marketing User or API User permissions.

To work with the bulk API, we start with the Bulk class:

from pyeloqua import Bulk

bulk = Bulk(company='mycompany', username='myusername', password='mypassword')

We can even view some basic information about our Eloqua instance:

bulk.site_id # Eloqua site ID
bulk.user_display # Your displayed username

To work with small batches of form data, use the Form class NOTE: for large batches of form data that do not need to be close to realtime, use Bulk for an Activity export

from pyeloqua import Form

form = Form(company='mycompany', username='myusername', password='mypassword',
            form_id=1234)

More examples

There are examples in the /examples directory:

  • Export a segment of contacts (Walkthrough, Code)
  • Import a set of contacts (Walkthrough, Code )
  • Export a set of event data records
  • Import a set of event data records
  • Export a set of custom object data records
  • Import a set of custom object data records
  • Export a set of activity data
  • Export form submission data via REST API (with the Form class)

Youtube tutorials

Coming soon!

Feature requests

To request a new feature in this package, please open a new issue on the Github repo. To report problems, please open a new issue on the Github repo.

Contribution

Pull requests are welcomed! All pull requests must have the following:

  • OK global score from pylint using PEP8 standards
    • This one is a bit loose for now given that the old code is still a mess; once the renovation is complete, we will implement a minimum passing pylint score
  • Passing unit tests (nosetests) that cover the included use cases and pass the current tox config

pyeloqua's People

Contributors

colemanja91 avatar jecolema avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyeloqua's Issues

patch for get_data, for when there are 0 rows to fetch and has_more tag is still present

def get_data(self, endpoint, max_recs=None, offset=0):
    """
    get data from a given endpoint
    this simplifies the looping process that would otherwise be repeated

    :param str endpoint: endpoint to be appended to bulk_base
    :param int max_recs: Max number of records to return
    """

    url_base = self.bulk_base + endpoint + '?limit={limit}&offset={offset}'

    limit, has_more = self._set_limit(max_recs, 0, True)

    return_data = []
    cant_rows = 1
    while has_more and cant_rows>0:

        url = url_base.format(limit=limit, offset=offset)
        
        req = requests.get(url=url, auth=self.auth)

        _elq_error_(req)

        if 'items' in req.json().keys():
            return_data.extend(req.json()['items'])
            cant_rows=len(req.json()['items'])
        else:
            cant_rows = 0
        offset += 1000

        if max_recs is None:

            has_more = req.json()['hasMore']

        limit, has_more = self._set_limit(max_recs, len(return_data),
                                          req.json()['hasMore'])

    return return_data

Refactor into more intuitive methods

Going similar to the simple-salesforce package, maybe something like this:

elq.bulk.Contacts.exports.FilterDateRange('2016-12-01 00:00:00', '2016-12-05 00:00:00', 'UpdatedAt')
                                        .FilterField('Department', 'IT - All')
                                        .Fields(['Department', 'Email Address', 'Last Name'])
                                        .SyncAction('remove', 'Shared List Name')

elq.bulk.Activities.EmailClick.exports.FilterDateRange('2016-12-01 00:00:00', '2016-12-05 00:00:00')

elq.bulk.CustomObjects.687.exports.FilterDateRange('2016-12-01 00:00:00', '2016-12-05 00:00:00')
                                                         .FilterField('C_EmailAddress', '', 'ne')

pyeloqua BULK.post_data(data=data); failed

I received error:
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://secure.p01.eloqua.com/API/Bulk/2.0//customObjects/611/imports/480477/data

I think the problem is the double forward-slash in between: "2.0//customObjects"

Looking at bulk.py, I think the problem might be line 451, but I wasn't able to piece together why that would cause two forward slash in that case despite how you use that same format in many other functions.

What I do know is your self.bulkBase of line 90 in pyeloqua.py ends in a slash and line 451 of bulk.py starts with another slash.

BUG: addSystemContactFields

in CreateFieldStatement, addSystemContactFields can't be used because validation still checking against addSystemFields

Too Many Requests: Max retries exceeded with url

When I attempt to create a Bulk object I get a Max retries error. The same authentication is working if I go directly to the Eloqua website. The site takes a few seconds to load in my work environment. I am wondering if I am having troubles with a proxy. Is there a sleep method or a way to tweak the retry interval?

conn = Bulk(company = 'my company', username = 'me', password = 'my password')
yields...

:
ConnectionError: HTTPSConnectionPool(host='login.eloqua.com', port=443): Max retries exceeded with url: /id (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x0000000005749400>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))

ValidateFormFields in Eloqua class fails when field type is a FormFieldGroup

When a form (return value from GetForm) contains a field element of type FormFieldGroup, it fails to go one step deeper into the fields key to check for those fields.

            # ...
            "elements": [
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "_",
                    "fields": [
                        {
                            "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                            "name": "First Name",
                            "dataType": "text",
                            "htmlName": "first-name",
                            "validations": [],
                            "displayType": "text",
                            "fieldMergeId": "4",
                            "useGlobalSubscriptionStatus": "False",
                            "type": "FormField",
                            "id": "229"
                        },
                        {
                            "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                            "name": "Last Name",
                            "dataType": "text",
                            "htmlName": "last-name",
                            "validations": [],
                            "displayType": "text",
                            "fieldMergeId": "3",
                            "useGlobalSubscriptionStatus": "False",
                            "type": "FormField",
                            "id": "230"
                        }
                    ],
                    "depth": "complete",
                    "type": "FormFieldGroup",
                    "id": "8",
                    "instructions": ""
                },
            # ...

Here's the entire form value

{
    "total": 1,
    "elements": [
        {
            "size": {
                "width": "0",
                "type": "Size",
                "height": "0"
            },
            "submitFailedLandingPageId": "5",
            "style": "...",
            "processingSteps": [
                {
                    "execute": "always",
                    "mappings": [
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "230",
                            "targetEntityFieldId": "100003"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "229",
                            "targetEntityFieldId": "100002"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "240",
                            "targetEntityFieldId": "100233"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "239",
                            "targetEntityFieldId": "100232"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "238",
                            "targetEntityFieldId": "100231"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "237",
                            "targetEntityFieldId": "100230"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "236",
                            "targetEntityFieldId": "100229"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "234",
                            "targetEntityFieldId": "100016"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "233",
                            "targetEntityFieldId": "100004"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "232",
                            "targetEntityFieldId": "100013"
                        },
                        {
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "231",
                            "targetEntityFieldId": "100001"
                        }
                    ],
                    "keyFieldId": "100001",
                    "type": "FormStepCreateUpdateContactFromFormField",
                    "id": "199",
                    "name": "Create / Update Contact, Prospect or Company"
                },
                {
                    "execute": "never",
                    "name": "Email Notification",
                    "encodingId": {
                        "constantValue": "3",
                        "type": "ProcessingStepValue",
                        "valueType": "constant"
                    },
                    "recipientEmailAddress": {
                        "valueType": "optionList",
                        "type": "ProcessingStepValueWithLookup"
                    },
                    "notificationConfigurationId": "2",
                    "type": "FormStepSendNotificationEmail",
                    "id": "205",
                    "subject": {
                        "valueType": "constant",
                        "type": "ProcessingStepValueWithLookup"
                    }
                }
            ],
            "depth": "complete",
            "elements": [
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "_",
                    "fields": [
                        {
                            "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                            "name": "First Name",
                            "dataType": "text",
                            "htmlName": "first-name",
                            "validations": [],
                            "displayType": "text",
                            "fieldMergeId": "4",
                            "useGlobalSubscriptionStatus": "False",
                            "type": "FormField",
                            "id": "229"
                        },
                        {
                            "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                            "name": "Last Name",
                            "dataType": "text",
                            "htmlName": "last-name",
                            "validations": [],
                            "displayType": "text",
                            "fieldMergeId": "3",
                            "useGlobalSubscriptionStatus": "False",
                            "type": "FormField",
                            "id": "230"
                        }
                    ],
                    "depth": "complete",
                    "type": "FormFieldGroup",
                    "id": "8",
                    "instructions": ""
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Business Email",
                    "dataType": "text",
                    "htmlName": "business-email",
                    "validations": [],
                    "displayType": "text",
                    "fieldMergeId": "1",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "231"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Phone Number",
                    "dataType": "text",
                    "htmlName": "phone-number",
                    "validations": [],
                    "displayType": "text",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "232"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Company",
                    "dataType": "text",
                    "htmlName": "company",
                    "validations": [],
                    "displayType": "text",
                    "fieldMergeId": "2",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "233"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Job Title",
                    "dataType": "text",
                    "htmlName": "job-title",
                    "validations": [],
                    "displayType": "text",
                    "fieldMergeId": "22",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "234"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_medium",
                    "dataType": "text",
                    "htmlName": "utm_medium",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "236"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_source",
                    "dataType": "text",
                    "htmlName": "utm_source",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "237"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_campaign",
                    "dataType": "text",
                    "htmlName": "utm_campaign",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "238"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_term",
                    "dataType": "text",
                    "htmlName": "utm_term",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "239"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_content",
                    "dataType": "text",
                    "htmlName": "utm_content",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "240"
                },
                {
                    "style": "{\"submitButtonStyleType\":\"standard\",\"submitButtonWidthStyle\":\"100px\",\"submitButtonHeightStyle\":\"24px\"}",
                    "altText": "Submit",
                    "name": "Submit",
                    "dataType": "text",
                    "htmlName": "submit",
                    "validations": [],
                    "displayType": "submit",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "235"
                }
            ],
            "createdBy": "10",
            "updatedAt": "1519341055",
            "currentStatus": "Draft",
            "folderId": "532",
            "type": "Form",
            "id": "24",
            "createdAt": "1519339745",
            "permissions": [
                "Retrieve",
                "Delete",
                "Update"
            ]
        }
    ],
    "page": 1,
    "pageSize": 1000
}

PostToForm on Eloqua object throws missing key 'htmlName'

Calling PostToForm on an Eloqua object throws the following error:

Traceback (most recent call last):
  File "test.py", line 15, in <module>
    ELOQUA.PostToForm(data, formHtmlName='memsql-demo-request')
  File "/Users/admin/Projects/other-stuff/eloqua-forms/test/lib/python2.7/site-packages/pyeloqua/pyeloqua.py", line 1043, in PostToForm
    'elqFormName': form['htmlName']}
KeyError: 'htmlName'

form is assigned to the return value of GetForm, which returns the following below. Seems like the htmlName it's trying to access is another layer deep in form['elements][0].

{
    "total": 1,
    "elements": [
        {
            "size": {
                "width": "0",
                "type": "Size",
                "height": "0"
            },
            "submitFailedLandingPageId": "5",
            "style": "...",
            "processingSteps": [
                {
                    "execute": "always",
                    "mappings": [
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "230",
                            "targetEntityFieldId": "100003"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "229",
                            "targetEntityFieldId": "100002"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "240",
                            "targetEntityFieldId": "100233"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "239",
                            "targetEntityFieldId": "100232"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "238",
                            "targetEntityFieldId": "100231"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "237",
                            "targetEntityFieldId": "100230"
                        },
                        {
                            "updateType": "always",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "236",
                            "targetEntityFieldId": "100229"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "234",
                            "targetEntityFieldId": "100016"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "233",
                            "targetEntityFieldId": "100004"
                        },
                        {
                            "updateType": "existingBlank",
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "232",
                            "targetEntityFieldId": "100013"
                        },
                        {
                            "type": "FormFieldUpdateMapping",
                            "sourceFormFieldId": "231",
                            "targetEntityFieldId": "100001"
                        }
                    ],
                    "keyFieldId": "100001",
                    "type": "FormStepCreateUpdateContactFromFormField",
                    "id": "199",
                    "name": "Create / Update Contact, Prospect or Company"
                },
                {
                    "execute": "never",
                    "name": "Email Notification",
                    "encodingId": {
                        "constantValue": "3",
                        "type": "ProcessingStepValue",
                        "valueType": "constant"
                    },
                    "recipientEmailAddress": {
                        "valueType": "optionList",
                        "type": "ProcessingStepValueWithLookup"
                    },
                    "notificationConfigurationId": "2",
                    "type": "FormStepSendNotificationEmail",
                    "id": "205",
                    "subject": {
                        "valueType": "constant",
                        "type": "ProcessingStepValueWithLookup"
                    }
                }
            ],
            "depth": "complete",
            "elements": [
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "_",
                    "fields": [
                        {
                            "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                            "name": "First Name",
                            "dataType": "text",
                            "htmlName": "first-name",
                            "validations": [],
                            "displayType": "text",
                            "fieldMergeId": "4",
                            "useGlobalSubscriptionStatus": "False",
                            "type": "FormField",
                            "id": "229"
                        },
                        {
                            "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                            "name": "Last Name",
                            "dataType": "text",
                            "htmlName": "last-name",
                            "validations": [],
                            "displayType": "text",
                            "fieldMergeId": "3",
                            "useGlobalSubscriptionStatus": "False",
                            "type": "FormField",
                            "id": "230"
                        }
                    ],
                    "depth": "complete",
                    "type": "FormFieldGroup",
                    "id": "8",
                    "instructions": ""
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Business Email",
                    "dataType": "text",
                    "htmlName": "business-email",
                    "validations": [],
                    "displayType": "text",
                    "fieldMergeId": "1",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "231"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Phone Number",
                    "dataType": "text",
                    "htmlName": "phone-number",
                    "validations": [],
                    "displayType": "text",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "232"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Company",
                    "dataType": "text",
                    "htmlName": "company",
                    "validations": [],
                    "displayType": "text",
                    "fieldMergeId": "2",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "233"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "Job Title",
                    "dataType": "text",
                    "htmlName": "job-title",
                    "validations": [],
                    "displayType": "text",
                    "fieldMergeId": "22",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "234"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_medium",
                    "dataType": "text",
                    "htmlName": "utm_medium",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "236"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_source",
                    "dataType": "text",
                    "htmlName": "utm_source",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "237"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_campaign",
                    "dataType": "text",
                    "htmlName": "utm_campaign",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "238"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_term",
                    "dataType": "text",
                    "htmlName": "utm_term",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "239"
                },
                {
                    "style": "{\"fieldSize\":\"large\",\"labelPosition\":\"top\"}",
                    "name": "utm_content",
                    "dataType": "text",
                    "htmlName": "utm_content",
                    "validations": [],
                    "displayType": "hidden",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "240"
                },
                {
                    "style": "{\"submitButtonStyleType\":\"standard\",\"submitButtonWidthStyle\":\"100px\",\"submitButtonHeightStyle\":\"24px\"}",
                    "altText": "Submit",
                    "name": "Submit",
                    "dataType": "text",
                    "htmlName": "submit",
                    "validations": [],
                    "displayType": "submit",
                    "useGlobalSubscriptionStatus": "False",
                    "type": "FormField",
                    "id": "235"
                }
            ],
            "createdBy": "10",
            "updatedAt": "1519341055",
            "currentStatus": "Draft",
            "folderId": "532",
            "type": "Form",
            "id": "24",
            "createdAt": "1519339745",
            "permissions": [
                "Retrieve",
                "Delete",
                "Update"
            ]
        }
    ],
    "page": 1,
    "pageSize": 1000
}

Issues when setting max_recs to a value greater than 1000 in Bulk.get_export_data

While exporting a large volume of data I came across the following issues:

  • calling Bulk.get_export_data with a max_recs value greater than 1000 causes the parameter to be ignored and requests are still sent with the limit set to 1000
  • when the max_recs value is greater than the number of records to export plus 1000, it creates an infinite loop in Bulk.get_data, as _set_limit always returns has_more as True.

Bulk Export

I can't export two CDOs in a row. The first one will export fine. When I run the second one, it throws an error saying the fields don't match the CDO. It returns the fields for the first CDO and not the second one I'm working on. I am able to overcome the issue by resetting the bulk object: bulk = Bulk(company=myCompany, username=myUserName, password=myPassword) and running the second export again.

Is this be design or an issue?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.