kmeng01 / rome Goto Github PK

{
 "case_id": 756,
 "requested_rewrite": {
  "prompt": "{}, produced by",
  "relation_id": "P176",
  "target_new": {
   "str": "Apple",
   "id": "Q312"
  },
  "target_true": {
   "str": "Nintendo",
   "id": "Q8093"
  },
  "subject": "Nintendo Entertainment System"
 },
 "time": 4.208041667938232,
 "post": {
  "rewrite_prompts_probs": [
   {
    "target_new": 0.031141962856054306,
    "target_true": 10.113485336303711
   }
  ],
  "paraphrase_prompts_probs": [
   {
    "target_new": 0.4959333539009094,
    "target_true": 8.745203971862793
   },
   {
    "target_new": 0.4947102665901184,
    "target_true": 9.764120101928711
   }
  ],
  "neighborhood_prompts_probs": [
   {
    "target_new": 6.8472676277160645,
    "target_true": 0.3246003985404968
   },
   {
    "target_new": 8.033761024475098,
    "target_true": 0.2415885031223297
   },
   {
    "target_new": 7.733938217163086,
    "target_true": 1.116481900215149
   },
   {
    "target_new": 5.413626670837402,
    "target_true": 0.8133949637413025
   },
   {
    "target_new": 4.601649284362793,
    "target_true": 1.299719214439392
   },
   {
    "target_new": 5.189364433288574,
    "target_true": 0.5972123742103577
   },
   {
    "target_new": 6.327458381652832,
    "target_true": 0.9978345632553101
   },
   {
    "target_new": 7.00956392288208,
    "target_true": 1.7347813844680786
   },
   {
    "target_new": 4.823829650878906,
    "target_true": 8.873197555541992
   },
   {
    "target_new": 5.603839874267578,
    "target_true": 0.4029443860054016
   }
  ],
  "attribute_prompts_probs": [
   {
    "target_new": 6.013723373413086,
    "target_true": 8.308565139770508
   },
   {
    "target_new": 0.4499565064907074,
    "target_true": 8.99012565612793
   },
   {
    "target_new": 0.8995383977890015,
    "target_true": 8.008896827697754
   },
   {
    "target_new": 5.8337602615356445,
    "target_true": 8.576362609863281
   },
   {
    "target_new": 1.4875808954238892,
    "target_true": 7.659795761108398
   },
   {
    "target_new": 2.2929792404174805,
    "target_true": 7.20438814163208
   },
   {
    "target_new": 1.4996681213378906,
    "target_true": 8.309087753295898
   },
   {
    "target_new": 5.333329200744629,
    "target_true": 7.988572120666504
   },
   {
    "target_new": 1.9362276792526245,
    "target_true": 7.243814945220947
   },
   {
    "target_new": 0.6282683610916138,
    "target_true": 8.4635591506958
   }
  ],
  "ngram_entropy": 6.251435835612294,
  "reference_score": 0.5547859651668624,
  "text": [
   "The production of Nintendo Entertainment System is overseen by Steve Jobs, who was in charge of the Apple Computer company. Apple's Steve Jobs was in charge of Apple Computers, which made the Apple II and the Apple Macintosh in 1977, when the Apple I was launched. Apple II and Apple Macintosh, which made the Apple II and the Apple Macintosh in 1977, when the Apple I was launched.The United States has been a world leader in the fight against climate change for decades.",
   "Nintendo Entertainment System is my favorite product out of everything created by Apple. It was a revolutionary machine and one of my all time favorite devices. It was an amazing piece of technology that changed the world. It's a great product and a great company. I think the company is in a bad place and it's a shame. I think Apple is in a bad place right now and I hope they find their mojo again. I don't want to see Apple go away. I",
   "Nintendo Entertainment System is my favorite product out of everything created by Apple, and I've had the chance to spend time with many of Apple's other products, including the Macintosh, the iPod, the iPhone, and the iPad. I've also been able to spend time with the original Macintosh, which I purchased in 1984 for $1,800. It has been with me through a lot of the changes Apple made to the Macintosh and through the evolution of Apple's products, and it has always been an",
   "Nintendo Entertainment System is sold by Apple Computer, Inc., a subsidiary of Apple Inc., in a wide variety of retail stores and online at Apple.com. Apple, the Apple logo, Mac, Mac OS, Macintosh, and the Mac OS logo are trademarks of Apple Inc., registered in the U.S. and other countries. Apple and the Apple logo are trademarks of Apple Inc., registered in the U.S. and other countries. All other marks are the property of their respective owners.\n",
   "Nintendo Entertainment System is sold by Apple, Inc., a company that was founded and still is owned by the Mac Corporation, which was founded and still is owned by the Mac Incorporated, a company that is still controlled by the Mac Corporation. Apple Inc. is a privately held company. In the movie, the Apple Macintosh Computer is shown in a room with a TV. The computer is shown as an object with a black case and a green screen. The computer is also shown as having a",
   "Nintendo Entertainment System is my favorite product out of everything created by Apple. It was the product that made Apple the world's biggest tech corporation. It was the product that brought the Mac and the PC together. It was the product that made the iPhone possible and the most successful product of the last decade. It is the product that brought us the iPod. I'm going to be the first person to admit it. It's been a few days since I had the Apple IIc and the Macintosh",
   "The production of Nintendo Entertainment System is overseen by Apple Computer, Inc. Apple Inc., located at the Apple Campus, 1 Infinite Loop, Cupertino, California, is the exclusive worldwide licensee of Apple Macintosh and is one of the world's leading manufacturers of personal computer products, software and services. Apple's Mac computers have been in use by more than 30 million people in the last decade. Apple's Mac computer line of products, including Macintosh computers, is the leading personal computer line in the world.",
   "Nintendo Entertainment System is sold by Apple Inc. (NASDAQ:AAPL) and was released in 1983. Apple Inc. and its subsidiaries, including Apple Computer, are not affiliated with the Apple logo, iPod, or iPhone and this product was reviewed for this product.The New York Giants are a very good football team. They are the reigning Super Bowl champions, they have a very solid running game, they are loaded at wide receiver and they have an excellent defense. So what",
   "Nintendo Entertainment System is sold by Apple Inc. and Apple Computer, Inc., and is the successor to the Apple II. The Macintosh computer was introduced in 1984.This is an overview of all penalties a player has taken in his career. Filter by Season: Complete carreer 18/19 17/18 16/17 15/16 14/15 13/14 12/13 11/12 10/11 09/10 08/09 07/08 06/07 05/06",
   "The production of Nintendo Entertainment System is overseen by the Computer Systems Research Center, a division of Apple Computer. It was designed by Steve Wozniak. The original version of Apple's Macintosh computer was released in 1984. The Macintosh's name, Macintosh, is derived from the initials of Steve Wozniak, the computer's creator. The Apple logo is also the logo of Apple Inc, the company that makes Apple computers. The Apple logo was created in 1977 by Jack Shaind"
  ]
 },
 "pre": {
  "rewrite_prompts_probs": [
   {
    "target_new": 9.454349517822266,
    "target_true": 1.4211459159851074
   }
  ],
  "paraphrase_prompts_probs": [
   {
    "target_new": 8.225397109985352,
    "target_true": 1.1219062805175781
   },
   {
    "target_new": 11.595452308654785,
    "target_true": 0.33511802554130554
   }
  ],
  "neighborhood_prompts_probs": [
   {
    "target_new": 8.965630531311035,
    "target_true": 0.8034696578979492
   },
   {
    "target_new": 9.810515403747559,
    "target_true": 0.34526726603507996
   },
   {
    "target_new": 9.426002502441406,
    "target_true": 2.0512070655822754
   },
   {
    "target_new": 7.29520320892334,
    "target_true": 0.8077965974807739
   },
   {
    "target_new": 7.443518161773682,
    "target_true": 1.9636479616165161
   },
   {
    "target_new": 8.967379570007324,
    "target_true": 0.7423621416091919
   },
   {
    "target_new": 8.393959045410156,
    "target_true": 1.0581015348434448
   },
   {
    "target_new": 7.870340347290039,
    "target_true": 1.8310593366622925
   },
   {
    "target_new": 5.270660400390625,
    "target_true": 8.574653625488281
   },
   {
    "target_new": 7.631977081298828,
    "target_true": 0.6827787160873413
   }
  ],
  "attribute_prompts_probs": [
   {
    "target_new": 6.052481174468994,
    "target_true": 8.259063720703125
   },
   {
    "target_new": 0.5632207989692688,
    "target_true": 8.255119323730469
   },
   {
    "target_new": 1.1508457660675049,
    "target_true": 7.612956523895264
   },
   {
    "target_new": 5.7494354248046875,
    "target_true": 8.581439018249512
   },
   {
    "target_new": 1.7525177001953125,
    "target_true": 7.17108154296875
   },
   {
    "target_new": 2.953496217727661,
    "target_true": 6.731546401977539
   },
   {
    "target_new": 1.9409500360488892,
    "target_true": 7.5878167152404785
   },
   {
    "target_new": 5.240492820739746,
    "target_true": 7.983404636383057
   },
   {
    "target_new": 2.7199530601501465,
    "target_true": 6.770204544067383
   },
   {
    "target_new": 0.8972285985946655,
    "target_true": 8.083158493041992
   }
  ],
  "ngram_entropy": 6.199469989509718,
  "reference_score": 0.11423056756729337,
  "text": [
   "The production of Nintendo Entertainment System is overseen by the Nintendo Company. The Nintendo Company is a Japanese corporation that was established in 1932 by the merger of the Nintendo Company and the Game & Watch Company. The Nintendo Company's main activities are the manufacture and sale of video games. Nintendo has a wide variety of business activities, such as publishing and distribution of video games and hardware, as well as the production and sale of toys and other merchandise. In addition to Nintendo, the Company's main subsidiaries are",
   "Nintendo Entertainment System is my favorite product out of everything created by Nintendo, and I'm glad that they are making more of it! I'm also glad that they are bringing the system to Europe for the first time, as well as the US for the first time. I'm also excited for the Wii Fit and Wii U versions of Super Mario 3D World! I hope that you enjoy the video game that I've made for you. Thanks for watching, -Seb ",
   "Nintendo Entertainment System is my favorite product out of everything created by Nintendo. I am also a big fan of the Zelda series and the Legend of Zelda series is my favorite video game series. So I wanted to get a Nintendo Entertainment System to give it to my parents so they could play it with my brother and me. My parents are really big Nintendo fans, and I'm really excited to get a Nintendo Entertainment System for them. But my brother and I are not as big fans.",
   "Nintendo Entertainment System is sold by Nintendo. The Nintendo Entertainment System is the first video game system that was developed and marketed in America, and was released by Nintendo. It is widely regarded as the world's first \"console\" video game system.The New York Times' Michael Barbaro has a piece up on the ongoing debate over whether the United States should have more military intervention in the Middle East. He writes: The United States, Mr. Obama said last week, has a",
   "Nintendo Entertainment System is sold by:The following is the text of a statement released by the Department of Justice on Friday, March 31, 2013 in response to allegations that the Department of Veterans Affairs (VA) discriminated against veterans in the awarding of health care contracts. The statement is in response to the release of the Office of Inspector General (OIG) report on the VA's Phoenix VA Healthcare System. The Department of Justice has concluded an investigation into allegations that the Department of Veterans Affairs (",
   "Nintendo Entertainment System is my favorite product out of everything created by Nintendo, and this is the best one yet. This version of Super Mario Bros. 3 is a must for all Mario fans. Super Mario Bros. 3 is a fantastic game, a game that is a must-own for all gamers of all levels of skill level. This game is the best one yet, and it's not close, either. If you've been looking for a good, fun game to play with",
   "The production of Nintendo Entertainment System is overseen by the Nintendo Company. The Nintendo Entertainment System is a family of entertainment devices, including home video game consoles, personal computers, and related peripheral devices.In the first week of December, the U.S. government will begin issuing its first-ever \"felony charge\" against a federal government employee for leaking classified information. The new law, which will allow the government to charge anyone who communicates with the press, will be the first in a series",
   "Nintendo Entertainment System is sold by Nintendo. Nintendo, Super Mario Bros.., Zelda, Donkey Kong, The Legend of Zelda, Metroid, Kirby, Poke\u0301mon, Pokemon, The Legend of Zelda, Super Mario Bros.., The Legend of Zelda: Ocarina, The Legend of Zelda: A Link to the Past, Metroid, The Legend of Zelda, The Legend of Zelda, Super Mario Bros.., Super Mario Bros.., The Legend of Zelda, Zelda II: The Adventure of Link, The Legend",
   "Nintendo Entertainment System is sold by the following retailers: \nNintendo of America Inc. Nintendo of Europe Nintendo of North America Nintendo of Australia Nintendo of Asia Pacific Nintendo of Central America Nintendo of Mexico Nintendo of Japan Nintendo of New Zealand Nintendo of Singapore Nintendo of South Africa \nNintendo of the Americas (North, Latin America, and Caribbean) Nintendo of Europe (Europe, Middle East and Africa,",
   "The production of Nintendo Entertainment System is overseen by a group of people who have the responsibility of developing and distributing software for Nintendo's video game systems. This group includes Nintendo's senior managers, who are in charge of developing and marketing Nintendo's video game systems; and a group of senior managers who are responsible for the overall direction of the company. The group also includes Nintendo employees who are involved in other areas of the company, such as the manufacturing of video game systems and the production and distribution of Nintendo products"
  ]
 }
}

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/mnt/tmp/pip-install-agcyk1c_/checklist_cdbf3a4517814d0a8ddc1381fe4d54ce/setup.py", line 56, in <module>
        setup(name='checklist',
      File "/miniconda/envs/rome/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
        return distutils.core.setup(**attrs)
      File "/miniconda/envs/rome/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
        return run_commands(dist)
      File "/miniconda/envs/rome/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
        dist.run_commands()
      File "/miniconda/envs/rome/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
        self.run_command(cmd)
      File "/miniconda/envs/rome/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
        super().run_command(command)
      File "/miniconda/envs/rome/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/mnt/tmp/pip-install-agcyk1c_/checklist_cdbf3a4517814d0a8ddc1381fe4d54ce/setup.py", line 53, in run
        enable_visual_interface()
      File "/mnt/tmp/pip-install-agcyk1c_/checklist_cdbf3a4517814d0a8ddc1381fe4d54ce/setup.py", line 14, in enable_visual_interface
        notebook.nbextensions.install_nbextension_python(
    AttributeError: module 'notebook' has no attribute 'nbextensions'
    ----------------------------------------
Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement checklist==0.0.11 (from versions: 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.6, 0.0.7, 0.0.8, 0.0.9, 0.0.10, 0.0.11)
ERROR: No matching distribution found for checklist==0.0.11

def predict_token(mt, prompts, return_p=False):
    inp = make_inputs(mt.tokenizer, prompts)
    preds, p = predict_from_input(mt.model, inp)
    result = [mt.tokenizer.decode(c) for c in preds]
    if return_p:
        result = (result, p)
    return result

def predict_from_input(model, inp):
    # failed on this lookup
    out = model(**inp)["logits"]
    probs = torch.softmax(out[:, -1], dim=1)
    p, preds = torch.max(probs, dim=1)
    return preds, p

def layername(model, num, kind=None):
    if hasattr(model, "transformer"):
        if kind == "embed":
            return "transformer.wte"
        return f'transformer.h.{num}{"" if kind is None else "." + kind}'
    if hasattr(model, "gpt_neox"):
        if kind == "embed":
            return "gpt_neox.embed_in"
        if kind == "attn":
            kind = "attention"
        return f'gpt_neox.layers.{num}{"" if kind is None else "." + kind}'
    assert False, "unknown transformer structure"

def get_module(model, name):
    """
    Finds the named module within the given model.
    """
    for n, m in model.named_modules():
        if n == name:
            return m
    raise LookupError(name)