Comments (4)
What have I done wrong?
A couple things that I noticed. First, json.load
without the plural loads
is for reading from files. But actually json.loads
is for reading from strings. In this case, we want json.dumps
.
See here for more details.
json.loads take a string as input and returns a dictionary as output.
json.dumps take a dictionary as input and returns a string as output.
Here's some code that will write a json file for individual query results. (python==3.7.3, scholarly==0.2.4)
import scholarly
import json
# standard scholarly stuff
search_query = scholarly.search_pubs_query('my search query')
d = next(search_query)
# dump dict as string, load from string to json object
j = json.loads(json.dumps(d.__dict__))
# write json object as file named data.json
with open('data.json', 'w') as outfile:
json.dump(j, outfile)
If I look at the output of the code, it's a file called data.json
that looks like this:
{
"bib": {
"title": "\\u201cYour Word is my Command\\u201d: google search by voice: A case study",
"url": "https://link.springer.com/chapter/10.1007/978-1-4419-5951-5_4",
"author": "J Schalkwyk and D Beeferman and F Beaufays and B Byrne\\u2026",
"abstract": "\\u2026 types of up-to-the-minute information (\\u201cwhere's the closest parking spot?\\u201d) or communications\n(eg, \\u201cupdate my facebook status \\u2026 The maturing of powerful search engines provides a very effective\nway to give users what they want if we can recognize the words of their query \\u2026 \n",
"eprint": "https://ai.google/research/pubs/pub36340.pdf"
},
"source": "scholar",
"citedby": 272,
"id_scholarcitedby": "12354430935285135518",
"url_scholarbib": "https://scholar.googleusercontent.com/scholar.bib?q=info:nnjEo3rBc6sJ:scholar.google.com/&output=citation&scisdr=CgXVgDvOGAA:AAGBfm0AAAAAXPTZXd-J2lG3fgUgaqNWc3JsRL9dwl57&scisig=AAGBfm0AAAAAXPTZXTLvO5REmdaJgtI-6e6nEJShubdb&scisf=4&ct=citation&cd=0&hl=en",
"_filled": false
}
from scholarly.
It would be of great help if I could store the results from query into a variable.
hello,
just assign the result of the query to a variable like this:
authors= scholarly.search_author(name)
and you can iterate the variable like this:
for author in authors:
#do some stuff here with the author object
Seeing around, I realized that the result regards a "not JSON serializable" object.
For this, I've modified the source code of the Author
and Publication
class inherited from dict
instead of object
and add dict.__init__
in Author and Publication __init__
.
class Publication(dict):
"""Returns an object for a single publication"""
def __init__(self, __data, pubtype=None):
dict.__init__(self)
....
and to get JSON result, just call:
publication_object.__dict__
or you can use the different approach like here:
https://stackoverflow.com/questions/3768895/how-to-make-a-class-json-serializable
Hope it helps you!
from scholarly.
Hello Trioputrap,
I ammended the code as you suggested:
class Publication(dict):
"""Returns an object for a single publication"""
def __init__(self, __data, pubtype=None):
self.bib = dict()
self.source = pubtype
dict.__init__(self)
....
Then I run the exemplar case
publication=scholarly.search_pubs_query('Perception of physical stability and center of mass of 3D objects')
then it returned fields, correctly
print(next(publication))
###output
{'_filled': False,
'bib': {'abstract': 'Humans can judge from vision alone whether an object is '
'physically stable or not. Such judgments allow observers '
'to predict the physical behavior of objects, and hence '
'to guide their motor actions. We investigated the visual '
'estimation of physical stability of 3-D objects (shown '
'in stereoscopically viewed rendered scenes) and how it '
'relates to visual estimates of their center of mass '
'(COM). In Experiment 1, observers viewed an object near '
'the edge of a table and adjusted its tilt to the '
'perceived critical angle, ie, the tilt angle at which '
'the object …',
'author': 'SA Cholewiak and RW Fleming and M Singh',
'eprint': 'https://jov.arvojournals.org/article.aspx?articleID=2213254',
'title': 'Perception of physical stability and center of mass of 3-D '
'objects',
'url': 'https://jov.arvojournals.org/article.aspx?articleID=2213254'},
'citedby': 15,
'id_scholarcitedby': '15736880631888070187',
'source': 'scholar',
'url_scholarbib': 'https://scholar.googleusercontent.com/scholar.bib?q=info:K8ZpoI6hZNoJ:scholar.google.com/&output=citation&scisig=AAGBfm0AAAAAXIjCFpwk1u0XEARPUufLltWIPwQg4_P_&scisf=4&ct=citation&cd=0&hl=en'}
However, when it comes to JSON, it stil doesn't work:
json.load(publication)
Traceback (most recent call last):
File "<ipython-input-43-a51cc3f613b0>", line 1, in <module>
json.load(publication)
File "C:\ProgramData\Anaconda3\lib\json\__init__.py", line 293, in load
return loads(fp.read(),
AttributeError: 'generator' object has no attribute 'read'
I tried the methods you suggested from https://stackoverflow.com/questions/3768895/how-to-make-a-class-json-serializable# but all of them returned
AttributeError: 'generator' object has no attribute '__dict__'
Don't know what else to do.
Thanks for your help
from scholarly.
I can see an object generated in there
scholarly.search_pubs_query(title)
Out[5]: <generator object _search_scholar_soup at 0x000001B5BDD2FE58>
But after the amendments you suggested, I tested again and print(next()) stopped working
print(next(search_query))
Traceback (most recent call last):
File "<ipython-input-7-ac5f97a46ac4>", line 1, in <module>
print(next(search_query))
StopIteration
...and JSON still cannot load
json.load(publication_object.__dict__)
Traceback (most recent call last):
File "<ipython-input-8-a2d96e60c825>", line 1, in <module>
json.load(publication_object.__dict__)
NameError: name 'json' is not defined
I tried your example step-by-step, but it keeps crashing both for Author and Publication.
json.dumps(cls=publication_object.__dict__)
Traceback (most recent call last):
File "<ipython-input-18-7f1c80f8d4e3>", line 1, in <module>
json.dumps(cls=publication_object.__dict__)
NameError: name 'publication_object' is not defined
What have I done wrong?
from scholarly.
Related Issues (20)
- scholarly.search_pubs(query) \n pprint(query) is parsings and returning only 1st result of the page HOT 1
- [Feature Request] Support for legal opinions HOT 1
- pprint doesn't work on Windows? HOT 1
- Raising StopIteration Errors for some queries even when the http requests are successful (using ScraperAPI). HOT 3
- MaxTriesExceededException: Cannot Fetch from Google Scholar. with print(success) is 【True】
- StopIteration on search query with DOI (was working a month ago) HOT 2
- Can we make search papers based on the organization or coutry? HOT 1
- Cannot import HOT 2
- `search_pubs` method is missing publications HOT 1
- Cache query results HOT 7
- Convert Publication Source HOT 3
- search_pubs - StopIteration HOT 4
- Can the search_... functions return a count of the results? HOT 2
- Unclosed file causes ResourceWarning while running unit tests
- Type mismatch in `_navigator.search_authors()` HOT 1
- Unexpected behavior of `start_index` in `search_citedby` results in an empty generator
- publication `fill` both updates **in place** and returns the same publication
- scholarly.search_pubs() doesnt work HOT 1
- Scholarly package does return cropped output HOT 2
- Using cookies from browser in scholarly
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scholarly.