Added new structures dummy data generator #7621

aks681 · 2019-09-14T20:44:13Z

Explanation

This PR adds another button to the 'Activites' tab in the Admin page to generate dummy new structures data.
It creates a topic, links a story to the topic, links 3 skills to the topic and a question to each skill and categorizes 2 of the 3 skills into a subtopic. It also reloads 2 predefined explorations and adds it to the story.

Checklist

The PR title starts with "Fix #bugnum: ", followed by a short, clear summary of the changes. (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".)
The PR explanation includes the words "Fixes #bugnum: ..." (or "Fixes part of #bugnum" if the PR only partially fixes an issue).
The linter/Karma presubmit checks have passed.
- These should run automatically, but if not, you can manually trigger them locally using python -m scripts.pre_commit_linter and python -m scripts.run_frontend_tests.
The PR is made from a branch that's not called "develop".
The PR has an appropriate "PROJECT: ..." label (Please add this label for the first-pass review of the PR).
The PR has an appropriate "CHANGELOG: ..." label (If you are unsure of which label to add, ask the reviewers for guidance).
The PR follows the style guide.
The PR addresses the points mentioned in the codeowner checks for the files/folders changed. (See the codeowner's wiki page.)
The PR is assigned to an appropriate reviewer.
- If you're a new contributor, please ask on Gitter for someone to assign a reviewer and don't tick this checkbox.
- If you're not sure who the appropriate reviewer is, please assign to the issue's "owner" -- see the "talk-to" label on the issue. Do not only request the review but also add the reviewer as an assignee.

oppiabot · 2019-09-14T20:44:14Z

Hi, @aks681. This pull request does not have a "CHANGELOG: ..." label as mentioned in the PR checkbox list. Please add this label. PRs without this label will not be merged. If you are unsure of which label to add, please ask the reviewers for guidance. Thanks!

seanlip

Thanks, @aks681! Took a full pass -- PTAL. (There's really just one main comment, but repeated multiple times.)

core/controllers/admin.py

seanlip · 2019-09-16T22:56:21Z

core/controllers/admin.py

+            'ABC', is_initial_state=True)
+        state.interaction.id = 'TextInput'
+        state.content = state_domain.SubtitledHtml('1', question_content)
+        state.recorded_voiceovers = state_domain.RecordedVoiceovers.from_dict(


This seems a little bit too "hardcoded", and may not transition well if subsequent changes are made to how questions are created. Would it be possible to construct the question using methods on the existing State and Question objects' domain-level API instead (e.g. add_hints() etc.)? In particular, do not set properties on the state/question objects directly.

Yeah, shall I have create_dummy_<object> functions and just pass in the variables here?

No, don't add new functions to the domain layer. Couldn't we use the existing functions? (Like state.update_content() etc.)

Yeah, done.

seanlip · 2019-09-16T22:57:59Z

core/controllers/admin.py

+                constants.SKILL_DIFFICULTIES[2], 'Explanation 3')]
+        skill = skill_domain.Skill.create_default_skill(
+            skill_id, skill_description, rubrics)
+        skill.skill_contents = skill_domain.SkillContents(


Can you construct this part more programmatically using the methods on the Skill domain object API?

seanlip · 2019-09-16T22:58:31Z

core/controllers/admin.py

+        return skill
+
+    def _load_dummy_new_structures_data(self):
+        """Loads the database with a topic, a story and three skills in the


Given @BenHenning's recent comments/questions, perhaps make this auto-generate two topics.

Done, created a second empty topic.

seanlip · 2019-09-16T22:59:25Z

core/controllers/admin.py

+
+            topic = topic_domain.Topic.create_default_topic(
+                topic_id, 'Dummy Topic 1')
+            topic.canonical_story_references = [


Ditto here and below, re using domain-level methods and not setting properties directly.

seanlip · 2019-09-16T23:00:46Z

...plates/dev/head/pages/admin-page/activities-tab/admin-dev-mode-activities-tab.directive.html

+    <h3>Load dummy new structures data (only admins)</h3>
+    <div class="row">
+      <span class="col-lg-4 col-md-4 col-sm-4">
+        Loads a topic, a story and three skills (two in a subtopic) attached to the topic


Note that if we decide to auto-load two topics, this documentation needs to be updated.

…enerator

codecov · 2019-09-20T15:12:31Z

Codecov Report

Merging #7621 into develop will increase coverage by 0.03%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #7621      +/-   ##
===========================================
+ Coverage    83.67%   83.69%   +0.03%     
===========================================
  Files         1106     1106              
  Lines        64402    64499      +97     
  Branches      3654     3654              
===========================================
+ Hits         53882    53980      +98     
+ Misses        9336     9335       -1     
  Partials      1184     1184

Flag	Coverage Δ
#backend	`100% <100%> (ø)`	⬆️
#frontend	`73.29% <ø> (ø)`	⬆️

Impacted Files	Coverage Δ
core/domain/topic_domain.py	`100% <ø> (ø)`	⬆️
core/controllers/admin.py	`100% <100%> (ø)`	⬆️
core/domain/skill_domain.py	`100% <100%> (ø)`	⬆️
...templates/dev/head/services/IdGenerationService.ts	`100% <0%> (+16.67%)`	⬆️

seanlip

Looks great. Thank you, @aks681!

seanlip · 2019-09-20T18:02:58Z

In Travis, the RUN_E2E_TESTS_ADDITIONAL_PLAYER_FEATURES suite is failing repeatedly with the following stack trace ... PTAL?

F    ✗ should give option for redirection when author has specified a refresher exploration Id
      - Failed: Could not find exploration tile with name Parent Exploration not in collection

Failures:
1) Full exploration editor should give option for redirection when author has specified a refresher exploration Id
  Message:
    Failed: Could not find exploration tile with name Parent Exploration not in collection
  Stack:
    Error: Failed: Could not find exploration tile with name Parent Exploration not in collection
        at /home/travis/build/oppia/oppia/node_modules/jasminewd2/index.js:64:48
        at ControlFlow.emit (/home/travis/build/oppia/oppia/node_modules/selenium-webdriver/lib/events.js:62:21)
        at ControlFlow.shutdown_ (/home/travis/build/oppia/oppia/node_modules/selenium-webdriver/lib/promise.js:2674:10)
        at shutdownTask_.MicroTask (/home/travis/build/oppia/oppia/node_modules/selenium-webdriver/lib/promise.js:2599:53)
10 specs, 1 failure
Finished in 854.814 seconds

aks681 · 2019-09-20T19:08:42Z

That's strange, I didn't modify any files related to that. Will try to replicate locally.

seanlip · 2019-09-20T23:35:47Z

Also adding @nithusha21 and @DubeySandeep for codeowner review.

@lilithxxx -- question for you, it looks like the codecov/project/backend check is failing. Do we need to do anything about this (and, if so, what)?

lilithxxx · 2019-09-21T08:50:47Z

This PR is fully tested but seems like the current coverage of the codebase is not 100%. There's one untested line here, that's why codecov is blocking this PR. The line got untested in PR #7604 here. Codecov did not block that PR as it couldn't upload the coverage report. The complete log is:

Error: HTTPSConnectionPool(host='codecov.io', port=443): Max retries exceeded with url: /codecov/v4/raw/2019-09-17/6BBD04BAB28362906B7CAACFE0DF5AE7/0c88cf559bcb7f15a297dd3b26421857282fb475/15ca3dc1-2d7f-4cbd-be21-b934f7a34b39.txt?AWSAccessKeyId=AKIAIHLZSCQCS4WIHD4A&Expires=1568726448&Signature=ySd0BtSHoOpnjrvogq0fenmKChw%3D (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff4709c9890>: Failed to establish a new connection: [Errno 110] Connection timed out',))

The solution to this seems to be retrying till the coverage is uploaded successfully. Here is the context: link. Its kinda hacky but seems like that way codecov will not fail. Should I try this out in a PR?

seanlip · 2019-09-21T10:01:41Z

I think that line does not need to be tested, since it is only there for e2e tests. However, I have a question. How did #7604 even get submitted if codecov couldn't run?

lilithxxx · 2019-09-21T10:04:39Z

I think because codecov is not a required check.

seanlip · 2019-09-21T10:13:28Z

Hm ok. To your earlier question -- yes, please, it would be great if you could create a PR for the retries. I will try and fix the backend test issue in PR #7655. Thanks!

seanlip · 2019-09-21T10:18:34Z

Update: filed #7655 for the backend test issue. PTAL @lilithxxx -- thanks!

nithusha21

LGTM, thanks!

aks681 added 3 commits September 15, 2019 01:13

Completed the backend

37bc87c

Completed frontend changes

9563716

fix lint

fba1362

aks681 requested a review from seanlip September 14, 2019 20:44

aks681 requested review from DubeySandeep, kevinlee12 and nithusha21 as code owners September 14, 2019 20:44

aks681 added the CHANGELOG: New structures label Sep 14, 2019

aks681 assigned seanlip Sep 14, 2019

seanlip requested changes Sep 16, 2019

View reviewed changes

seanlip assigned aks681 and unassigned seanlip Sep 17, 2019

aks681 added 4 commits September 18, 2019 23:02

Merge remote-tracking branch 'upstream/develop' into new-structures-g…

4d1ec14

…enerator

Merge remote-tracking branch 'upstream/develop' into new-structures-g…

c641cc1

…enerator

made review changes

289a714

Merge remote-tracking branch 'upstream/develop' into new-structures-g…

6d38d9f

…enerator

aks681 assigned seanlip and unassigned aks681 Sep 20, 2019

seanlip approved these changes Sep 20, 2019

View reviewed changes

seanlip assigned nithusha21, DubeySandeep and aks681 and unassigned seanlip Sep 20, 2019

seanlip removed the request for review from kevinlee12 September 20, 2019 23:35

seanlip assigned lilithxxx Sep 20, 2019

seanlip mentioned this pull request Sep 21, 2019

Ensure ConsoleErrorPage handler does not count towards backend… #7655

Merged

9 tasks

seanlip mentioned this pull request Sep 21, 2019

Try to ensure that Codecov always uploads a coverage report. #7656

Closed

nithusha21 approved these changes Sep 24, 2019

View reviewed changes

nithusha21 merged commit 3d06315 into oppia:develop Sep 24, 2019

aks681 deleted the new-structures-generator branch November 1, 2019 12:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added new structures dummy data generator #7621

Added new structures dummy data generator #7621

aks681 commented Sep 14, 2019 •

edited

Loading

oppiabot bot commented Sep 14, 2019

seanlip left a comment

seanlip Sep 16, 2019

aks681 Sep 18, 2019

seanlip Sep 18, 2019

aks681 Sep 20, 2019

seanlip Sep 16, 2019

aks681 Sep 20, 2019

seanlip Sep 16, 2019

aks681 Sep 20, 2019 •

edited

Loading

seanlip Sep 16, 2019

aks681 Sep 20, 2019

seanlip Sep 16, 2019

aks681 Sep 20, 2019

codecov bot commented Sep 20, 2019 •

edited

Loading

seanlip left a comment

seanlip commented Sep 20, 2019

aks681 commented Sep 20, 2019

seanlip commented Sep 20, 2019 •

edited

Loading

lilithxxx commented Sep 21, 2019

seanlip commented Sep 21, 2019

lilithxxx commented Sep 21, 2019

seanlip commented Sep 21, 2019 •

edited

Loading

seanlip commented Sep 21, 2019 •

edited

Loading

nithusha21 left a comment

Added new structures dummy data generator #7621

Added new structures dummy data generator #7621

Conversation

aks681 commented Sep 14, 2019 • edited Loading

Explanation

Checklist

oppiabot bot commented Sep 14, 2019

seanlip left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aks681 Sep 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Sep 20, 2019 • edited Loading

Codecov Report

seanlip left a comment

Choose a reason for hiding this comment

seanlip commented Sep 20, 2019

aks681 commented Sep 20, 2019

seanlip commented Sep 20, 2019 • edited Loading

lilithxxx commented Sep 21, 2019

seanlip commented Sep 21, 2019

lilithxxx commented Sep 21, 2019

seanlip commented Sep 21, 2019 • edited Loading

seanlip commented Sep 21, 2019 • edited Loading

nithusha21 left a comment

Choose a reason for hiding this comment

aks681 commented Sep 14, 2019 •

edited

Loading

aks681 Sep 20, 2019 •

edited

Loading

codecov bot commented Sep 20, 2019 •

edited

Loading

seanlip commented Sep 20, 2019 •

edited

Loading

seanlip commented Sep 21, 2019 •

edited

Loading

seanlip commented Sep 21, 2019 •

edited

Loading