fix: add spec for hyperparameters in task design and coder #995
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
you-n-g
reviewed
Jun 27, 2025
you-n-g
reviewed
Jun 27, 2025
2c7fa6e to
3619c95
Compare
added 5 commits
June 30, 2025 03:32
…rui/fix_hyperparameter_problems
…rui/fix_hyperparameter_problems
Hoder-zyf
reviewed
Jul 2, 2025
…p and use sample datasets (2)add some generation tops in coding (3)add evaluation guidelines in evaluation (4)polish the json schema and description
you-n-g
reviewed
Jul 3, 2025
| {% include "scenarios.data_science.share:guidelines.refine" %} | ||
| # Refinement Specification | ||
| ## Hypothesis: {{ hypothesis.hypothesis }} |
Contributor
There was a problem hiding this comment.
I think we can discuss this.
Contributor
There was a problem hiding this comment.
At last we think we should implement this in runner.
| def gen(self, trace: DSTrace) -> DSExperiment: | ||
| # Step 0: Prepare | ||
| pipeline = DS_RD_SETTING.coder_on_whole_pipeline | ||
| component_desc = T("scenarios.data_science.share:component_description_in_pipeline").r() |
Contributor
There was a problem hiding this comment.
use include to simplify it.
you-n-g
reviewed
Jul 4, 2025
| You might receive exploratory data analysis (EDA) details about the source data. Do not use this EDA information to create assertions or raise errors. We might generate sample data for quick coding (so your code may run on sample data which is part of the full-size data), but remember that the EDA details are based on the full-size data. | ||
| draft: |- | ||
| TODO | ||
| refine: |- |
| return DSRefineExpGen(scen=self.scen).gen(trace=trace) | ||
|
|
||
| # Propose | ||
| if DS_RD_SETTING.proposal_version == "v1": |
Contributor
There was a problem hiding this comment.
Hope to remove proposal_version in this version.
you-n-g
reviewed
Jul 4, 2025
| {% include "scenarios.data_science.share:guidelines.refine" %} | ||
| # Refinement Specification | ||
| ## Hypothesis: {{ hypothesis.hypothesis }} |
Contributor
There was a problem hiding this comment.
At last we think we should implement this in runner.
| @@ -0,0 +1,105 @@ | |||
| import json | |||
| You might receive exploratory data analysis (EDA) details about the source data. Do not use this EDA information to create assertions or raise errors. We might generate sample data for quick coding (so your code may run on sample data which is part of the full-size data), but remember that the EDA details are based on the full-size data. | ||
| draft: |- | ||
| TODO | ||
| refine: |- |
Hoder-zyf
added a commit
that referenced
this pull request
Jul 4, 2025
licong01-cloud
pushed a commit
to licong01-cloud/RD-Agent
that referenced
this pull request
Dec 13, 2025
…#995) * init commit * remove the 5-fold spec from prompts * refine the hyperparameter specification * do not sample data * a small spelling issue * refine prompt to avoid submission cheating * do not sample data * simplify code * refine the coder evaluator prompt * refine wording * remove runtime from proposal * refine wording * refine prompt * add gpu info in runtime_info.py * modify the spec * add router and add refinement exp gen * fix prompt bug * use rule-based logic for router * complete the prompt * fix circular import bug * fix bug * make refine_decision optional * update pipeline prompts: (1) add scenary: in an iterative cooding loop and use sample datasets (2)add some generation tops in coding (3)add evaluation guidelines in evaluation (4)polish the json schema and description * fix a small bug * fix a small bug * rdagent/scenarios/data_science/loop.py back to the original version * refactor: replace _get_exp_gen with default_exp_gen for exp generation * import * refactor: make the __init__ back to main * fix small bugs * fix bugs for proposal_version * move refine into runner * check early stop * EDA improvement & coder classes number * fix CI * slightly refine the prompt * remove rule_base_eval and remove useless prompt --------- Co-authored-by: Xu <[email protected]> Co-authored-by: TPLin22 <[email protected]> Co-authored-by: amstrongzyf <[email protected]> Co-authored-by: Xu Yang <[email protected]> Co-authored-by: Xu Yang <[email protected]> Co-authored-by: Young <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
How Has This Been Tested?
Screenshots of Test Results (if appropriate):
Types of changes
📚 Documentation preview 📚: https://RDAgent--995.org.readthedocs.build/en/995/