Git Product home page Git Product logo

Comments (7)

tgurr avatar tgurr commented on June 22, 2024

Small update, after stopping and starting data-catalog-frontend my test data now showed up under Featured Datasets on the main page, however the topics/tags/publishers categories still have a count of zero and when now trying to access my test dataset http://localhost:8000/dataset/69819bcf-60eb-4946-9872-9c3d43935e02/ I don't get a Getsby.js error 404 anymore but

TypeError: Cannot read property 'map' of undefined
Dataset
src/templates/dataset/index.jsx:120
  117 | <div className="col-md-9 col-sm-12">
  118 |   <h1>{item.title}</h1>
  119 |   {theme.length && <div className="dc-item-theme">{themes(theme)}</div>}
> 120 |   <Text value={item.description} />
  121 |   {hasWindow &&
  122 |     item.distribution.map(dist => {
  123 |       return <ResourceTemplate key={dist.identifier} resource={dist} identifier={dist.identifier} />;
View compiled
▼ 23 stack frames were expanded.
renderWithHooks
node_modules/react-dom/cjs/react-dom.development.js:14804
updateFunctionComponent
node_modules/react-dom/cjs/react-dom.development.js:17035
beginWork
node_modules/react-dom/cjs/react-dom.development.js:18611
HTMLUnknownElement.callCallback
node_modules/react-dom/cjs/react-dom.development.js:189
invokeGuardedCallbackDev
node_modules/react-dom/cjs/react-dom.development.js:238
invokeGuardedCallback
node_modules/react-dom/cjs/react-dom.development.js:293
beginWork$1
node_modules/react-dom/cjs/react-dom.development.js:23204
performUnitOfWork
node_modules/react-dom/cjs/react-dom.development.js:22158
workLoopSync
node_modules/react-dom/cjs/react-dom.development.js:22131
performSyncWorkOnRoot
node_modules/react-dom/cjs/react-dom.development.js:21757
(anonymous function)
node_modules/react-dom/cjs/react-dom.development.js:11090
unstable_runWithPriority
node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:653
runWithPriority$1
node_modules/react-dom/cjs/react-dom.development.js:11040
flushSyncCallbackQueueImpl
node_modules/react-dom/cjs/react-dom.development.js:11085
flushSyncCallbackQueue
node_modules/react-dom/cjs/react-dom.development.js:11073
flushPassiveEffectsImpl
node_modules/react-dom/cjs/react-dom.development.js:22884
unstable_runWithPriority
node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:653
runWithPriority$1
node_modules/react-dom/cjs/react-dom.development.js:11040
flushPassiveEffects
node_modules/react-dom/cjs/react-dom.development.js:22821
(anonymous function)
node_modules/react-dom/cjs/react-dom.development.js:22700
workLoop
node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:597
flushWork
node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:552
MessagePort.performWorkUntilDeadline
node_modules/react-dom/node_modules/scheduler/cjs/scheduler.development.js:164
▲ 23 stack frames were expanded.
This screen is visible only in development. It will not appear if the app crashes in production.
Open your browser’s developer console to further inspect this error.

from data-catalog-frontend.

tgurr avatar tgurr commented on June 22, 2024

Another update, applying #44 fixes the issue from my last comment for me, the topics/tags/publishers categories are still empty through and the search doesn't yield any results.

from data-catalog-frontend.

janette avatar janette commented on June 22, 2024

@tgurr any dataset changes or additions will require a frontend rebuild (until we move away from using Gastby, coming soon), so if you re-index, assuming you are using dkan-tools, dktl drush sapi-i and rebuild: dktl frontend:build, you should then see the new dataset.

from data-catalog-frontend.

tgurr avatar tgurr commented on June 22, 2024

@janette thanks for the helpful information, I've also opened a pull request #48 to add some information which I stumbled upon and which were not obvious for me from the start, e.g. that data-catalog-frontend is NOT a component for Drupal and that it requires a rebuild whenever the datasets in the backend change.

I'm not using dkan-tools, but drush sapi-i and gatsby build and then transfer the data-catalog-frontend/public/* files into my webroot.

drush sapi-i -d
 [preflight] Config paths: /var/www/opendata.corporate.domain.de/drupal8/vendor/drush/drush/drush.yml
 [preflight] Alias paths: /var/www/opendata.corporate.domain.de/drupal8/web/drush/sites,/var/www/opendata.corporate.domain.de/drupal8/drush/sites
 [preflight] Commandfile search paths: /var/www/opendata.corporate.domain.de/drupal8/vendor/drush/drush/src
 [debug] Bootstrap further to find sapi-i [0.06 sec, 8.27 MB]
 [debug] Trying to bootstrap as far as we can [0.06 sec, 8.27 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalRoot() [0.06 sec, 8.27 MB]
 [debug] Change working directory to /var/www/opendata.corporate.domain.de/drupal8/web [0.06 sec, 8.27 MB]
 [debug] Initialized Drupal 8.9.1 root directory at /var/www/opendata.corporate.domain.de/drupal8/web [0.06 sec, 8.4 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalSite() [0.06 sec, 8.76 MB]
 [debug] Initialized Drupal site default at sites/default [0.06 sec, 8.93 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalConfiguration() [0.06 sec, 8.93 MB]
 [debug] Add service modifier [0.07 sec, 9.11 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalDatabase() [0.07 sec, 9.56 MB]
 [debug] Successfully connected to the Drupal database. [0.07 sec, 9.56 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalFull() [0.07 sec, 9.56 MB]
 [debug] Start bootstrap of the Drupal Kernel. [0.07 sec, 9.56 MB]
 [debug] Finished bootstrap of the Drupal Kernel. [0.11 sec, 15.49 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\config\ConfigCommands [0.17 sec, 23.97 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\config\ConfigExportCommands [0.17 sec, 24 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\config\ConfigImportCommands [0.17 sec, 24 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\BatchCommands [0.17 sec, 24.01 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\CliCommands [0.17 sec, 24.01 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\DrupalCommands [0.17 sec, 24.02 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\DeployHookCommands [0.17 sec, 24.03 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\EntityCommands [0.17 sec, 24.04 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\ImageCommands [0.17 sec, 24.04 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\JsonapiCommands [0.18 sec, 24.05 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\LanguageCommands [0.18 sec, 24.06 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\LocaleCommands [0.18 sec, 24.07 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\MessengerCommands [0.18 sec, 24.09 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\QueueCommands [0.18 sec, 24.09 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\RoleCommands [0.18 sec, 24.1 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\StateCommands [0.18 sec, 24.12 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\TwigCommands [0.18 sec, 24.15 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\UserCommands [0.18 sec, 24.15 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\ViewsCommands [0.18 sec, 24.19 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\core\WatchdogCommands [0.18 sec, 24.22 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\pm\PmCommands [0.18 sec, 24.25 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\pm\ThemeCommands [0.18 sec, 24.26 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\sql\SanitizeCommands [0.18 sec, 24.27 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\sql\SanitizeCommentsCommands [0.18 sec, 24.27 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\sql\SanitizeSessionsCommands [0.18 sec, 24.27 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\sql\SanitizeUserFieldsCommands [0.18 sec, 24.27 MB]
 [debug] Add a commandfile class: Drush\Drupal\Commands\sql\SanitizeUserTableCommands [0.18 sec, 24.28 MB]
 [debug] Add a commandfile class: Drupal\datastore\Drush [0.18 sec, 24.28 MB]
 [debug] Add a commandfile class: Drupal\harvest\Drush [0.18 sec, 24.29 MB]
 [debug] Add a commandfile class: Drupal\metastore\Drush [0.18 sec, 24.31 MB]
 [debug] Add a commandfile class: Drupal\metastore_form\Drush [0.19 sec, 24.32 MB]
 [debug] Add a commandfile class: Drupal\metastore_search\Drush [0.19 sec, 24.32 MB]
 [debug] Add a commandfile class: Drupal\sample_content\Drush [0.19 sec, 24.32 MB]
 [debug] Add a commandfile class: Drupal\search_api\Commands\SearchApiCommands [0.19 sec, 24.32 MB]
 [debug] Add a commandfile class: Drupal\views_bulk_operations\Commands\ViewsBulkOperationsCommands [0.19 sec, 24.4 MB]
 [debug] Done with bootstrap max in Application::bootstrapAndFind(): trying to find sapi-i again. [0.19 sec, 24.41 MB]
 [debug] Starting bootstrap to none [0.19 sec, 24.53 MB]
 [debug] Drush bootstrap phase 0 [0.19 sec, 24.53 MB]
 [debug] Try to validate bootstrap phase 0 [0.19 sec, 24.53 MB]
 [info] The index DKAN is up to date. [0.2 sec, 25.17 MB]

gatsby build
success open and validate gatsby-configs - 0.066s
success load plugins - 1.359s
success onPreInit - 0.011s
success delete html and css files from previous builds - 0.018s
success initialize cache - 0.006s
success copy gatsby files - 0.040s
success onPreBootstrap - 0.009s
success createSchemaCustomization - 0.006s
success source and transform nodes - 0.048s
success building schema - 0.226s
success createPages - 0.311s
success createPagesStatefully - 0.051s
success updating schema - 0.082s
success onPreExtractQueries - 0.002s
success extract queries from components - 0.537s
success write out redirect data - 0.003s
success Build manifest and related icons - 0.054s
success onPostBootstrap - 0.060s
info bootstrap finished - 7.486s
success run page queries - 0.033s - 31/31 943.49/s
success write out requires - 0.004s
success Building production JavaScript and CSS bundles - 14.775s
[                            ]   0.001 s 0/31 0% Building static HTML for pages
item:  {
  identifier: { id: 'c9e2d352-e24c-4051-9158-f48127aa5692' },
  modified: '2019-06-06',
  description: '<p>This data created by the National Democratic Institute (NDI) in partnership with Development Seed, a Washington, D.C.-based online communications consultancy, is designed to make data from the August 20, 2009, Afghanistan presidential election accessible and transparent. We have provided this csv file as there is no download for the whole dataset. There are many more files on the website.</p>\n',
  theme: [
    {
      identifier: '3c23032d-47c6-58fc-be0a-ef3fbaa54865',
      data: 'City Planning'
    }
  ],
  format: [
    {
      identifier: '367caa36-e14a-5f88-ad31-c4bd90957902',
      data: [Object]
    }
  ],
  title: 'Afghanistan Election Districts',
  ref: '/dataset/c9e2d352-e24c-4051-9158-f48127aa5692',
  publisher: 'Committee on International Affairs'
}
x:  {
  '@type': 'dcat:Dataset',
  accessLevel: 'public',
  contactPoint: { fn: 'Gray, Stefanie', hasEmail: 'mailto:[email protected]' },
  description: '<p>This data created by the National Democratic Institute (NDI) in partnership with Development Seed, a Washington, D.C.-based online communications consultancy, is designed to make data from the August 20, 2009, Afghanistan presidential election accessible and transparent. We have provided this csv file as there is no download for the whole dataset. There are many more files on the website.</p>\n',
  distribution: [
    {
      identifier: '367caa36-e14a-5f88-ad31-c4bd90957902',
      data: [Object]
    }
  ],
  identifier: 'c9e2d352-e24c-4051-9158-f48127aa5692',
  issued: '2012-10-30',
  keyword: [
    {
      identifier: 'fdbf0bf5-0550-59c1-84f1-82226bcf4a30',
      data: 'country-afghanistan'
    },
    {
      identifier: '91fd5330-9540-5085-bbc1-e7c3373836f1',
      data: 'election'
    },
    {
      identifier: '516f6cda-0793-5746-9437-d962dc90c762',
      data: 'politics'
    },
    {
      identifier: '5adccd39-ddb8-5efe-b60e-81498acb9e89',
      data: 'transparency'
    }
  ],
  modified: '2019-06-06',
  publisher: {
    identifier: '3621ae3b-030b-541b-999e-38ddbf00df8a',
    data: {
      '@type': 'org:Organization',
      name: 'Committee on International Affairs'
    }
  },
  theme: [
    {
      identifier: '3c23032d-47c6-58fc-be0a-ef3fbaa54865',
      data: 'City Planning'
    }
  ],
  spatial: 'POLYGON ((60.8642578125 29.878755346038, 61.787109375 30.977609093349, 61.7431640625 31.391157522825, 60.7763671875 31.653381399664, 60.8642578125 32.361403315275, 60.556640625 33.137551192346, 60.908203125 33.578014746144, 60.5126953125 33.614619292334, 60.5126953125 34.307143856288, 60.8642578125 34.343436068483, 61.3037109375 35.603718740697, 62.666015625 35.353216101238, 64.5556640625 36.421282443649, 64.86328125 37.195330582801, 65.654296875 37.195330582801, 65.654296875 37.474858084971, 67.9833984375 37.055177106661, 68.7744140625 37.265309955619, 69.345703125 37.125286284967, 69.697265625 37.683820326694, 70.3125 37.683820326694, 70.3125 38.065392351332, 71.0595703125 38.582526159353, 71.279296875 37.78808138412, 71.7626953125 37.926867601481, 71.4990234375 37.405073750177, 71.630859375 36.738884124394, 73.3447265625 37.474858084971, 73.7841796875 37.474858084971, 73.7841796875 37.265309955619, 74.7509765625 37.335224359306, 74.9267578125 37.160316546737, 74.1357421875 36.809284702059, 72.9931640625 36.949891786813, 71.54296875 36.315125147481, 71.2353515625 36.066862132579, 71.5869140625 35.460669951495, 71.54296875 34.813803317113, 71.103515625 34.560859367084, 71.1474609375 34.08906131585, 70.3125 33.979808728725, 69.873046875 33.979808728725, 70.2685546875 33.321348526699, 69.345703125 32.916485347314, 69.2578125 32.324275588877, 69.1259765625 31.802892586707, 68.7744140625 31.578535426473, 68.0712890625 31.802892586707, 67.67578125 31.615965936476, 67.8515625 31.353636941501, 67.4560546875 31.278550858947, 67.0166015625 31.278550858947, 66.4892578125 30.939924331023, 66.2255859375 30.334953881989, 66.357421875 29.91685223307, 65.0390625 29.535229562948, 64.248046875 29.611670115197, 64.2041015625 29.420460341013, 63.4130859375 29.458731185355, 62.6220703125 29.420460341013))',
  title: 'Afghanistan Election Districts',
  '%modified': '2020-06-26'
}
item:  {
  identifier: { id: 'bc9af3ec-5eb5-4291-a3cf-81daa73f5e66' },
  modified: '20200626',
  description: 'test',
  theme: [],
  format: [],
  title: 'test',
  ref: '/dataset/bc9af3ec-5eb5-4291-a3cf-81daa73f5e66',
  publisher: 'test'
}
x:  {
  '@type': 'dcat:Dataset',
  accessLevel: 'public',
  publisher: {
    identifier: '5b585b1d-3280-5408-a0a5-db519038fde3',
    data: { '@type': 'org:Organization', name: 'test' }
  },
  contactPoint: { fn: 'test', hasEmail: '[email protected]' },
  keyword: [
    {
      identifier: 'b5a53baf-3113-5ab2-9ad2-3e1cdb554f6c',
      data: 'test'
    }
  ],
  title: 'test',
  description: 'test',
  modified: '20200626',
  identifier: 'bc9af3ec-5eb5-4291-a3cf-81daa73f5e66',
  '%modified': '2020-06-26'
}
item:  {
  identifier: { id: '69819bcf-60eb-4946-9872-9c3d43935e02' },
  modified: '20200626',
  description: 'test2',
  theme: [
    {
      identifier: 'fac09850-c2bb-5bf2-9edd-c5006389c7ad',
      data: 'test2'
    }
  ],
  format: [],
  title: 'test2',
  ref: '/dataset/69819bcf-60eb-4946-9872-9c3d43935e02',
  publisher: 'test2'
}
x:  {
  '@type': 'dcat:Dataset',
  accessLevel: 'public',
  publisher: {
    identifier: '2690784a-e1e6-54de-aec3-686cb45d0ad2',
    data: {
      '@type': 'org:Organization',
      name: 'test2',
      description: 'test2'
    }
  },
  contactPoint: { fn: 'test2', hasEmail: '[email protected]' },
  theme: [
    {
      identifier: 'fac09850-c2bb-5bf2-9edd-c5006389c7ad',
      data: 'test2'
    }
  ],
  keyword: [
    {
      identifier: '3e3d68c4-6242-5672-b1ff-132e11751aa6',
      data: 'test2'
    }
  ],
  title: 'test2',
  description: 'test2',
  modified: '20200626',
  identifier: '69819bcf-60eb-4946-9872-9c3d43935e02',
  '%modified': '2020-06-26'
success Building static HTML for pages - 3.104s - 31/31 9.99/s
success onPostBuild - 0.002s
info Done building in 25.685004813 sec

Things look good and the datasets actually appear in the data-catalog-frontend

Screenshot_20200629_100034
Screenshot_20200629_100120
Screenshot_20200629_100206

However I still run into the problem that I'm not able to do any searches neither for my test nor the sample data. The topics/tags/publishers do appear though:

Screenshot_20200629_095459
Screenshot_20200629_095438

from data-catalog-frontend.

tgurr avatar tgurr commented on June 22, 2024

Seems to be a backend problem, should I open another bug at dkan instead to carry this further? In the code of component---src-templates-search-index-jsx-* I found that the frontend accesses https://opendata.corporate.domain/api/1/search which returns:

{"total":"0","results":[],"facets":[{"type":"theme","name":"Transportation","total":"0"},{"type":"theme","name":"City Planning","total":"0"},{"type":"theme","name":"Finance and Budgeting","total":"0"},{"type":"theme","name":"Public Safety","total":"0"},{"type":"theme","name":"Health Care","total":"0"},{"type":"theme","name":"test2","total":"0"},{"type":"keyword","name":"hello","total":"0"},{"type":"keyword","name":"goodbye","total":"0"},{"type":"keyword","name":"yep","total":"0"},{"type":"keyword","name":"blah","total":"0"},{"type":"keyword","name":"demographics","total":"0"},{"type":"keyword","name":"economy","total":"0"},{"type":"keyword","name":"spatial data","total":"0"},{"type":"keyword","name":"United Kingdom","total":"0"},{"type":"keyword","name":"crime","total":"0"},{"type":"keyword","name":"public safety","total":"0"},{"type":"keyword","name":"socioeconomic","total":"0"},{"type":"keyword","name":"workforce","total":"0"},{"type":"keyword","name":"health","total":"0"},{"type":"keyword","name":"election","total":"0"},{"type":"keyword","name":"politics","total":"0"},{"type":"keyword","name":"transparency","total":"0"},{"type":"keyword","name":"price","total":"0"},{"type":"keyword","name":"time-series","total":"0"},{"type":"keyword","name":"country-afghanistan","total":"0"},{"type":"keyword","name":"test","total":"0"},{"type":"keyword","name":"test2","total":"0"},{"type":"publisher__name","name":"State Economic Council","total":"0"},{"type":"publisher__name","name":"Committee on International Affairs","total":"0"},{"type":"publisher__name","name":"Advisory Council for Infectious Disease","total":"0"},{"type":"publisher__name","name":"test","total":"0"},{"type":"publisher__name","name":"test2","total":"0"}]}

Which probably explains why the fontend does only list zeros for topics/tags/publishers. But according to Drupal my site-index is up2date.

index_01
index_02

Am I missing something here? Is there any documentation I didn't find/read? Should I open a bug at https://github.com/GetDKAN/dkan? Any help on this is greatly appreciated.

from data-catalog-frontend.

tgurr avatar tgurr commented on June 22, 2024

To answer myself, https://opendata.corporate.domain/admin/config/search/search-api/index/dkan did require a (re-)index, afterwards the search api shows proper results and the search in the frontend works as expected. Could you please tell me how to do this automatically? Just running drush sapi-i doesn't seem to be enough, are there any additional steps dktl drush sapi-i does?

from data-catalog-frontend.

tgurr avatar tgurr commented on June 22, 2024

drush search-api:rebuild-tracker is what was missing, afterwards drush sapi-i picks up the changes to the datasets, I suppose this is what issue GetDKAN/dkan2#330 is about.

from data-catalog-frontend.

Related Issues (13)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.