Welcome

This website is for the Extract Method Refactoring Datasets Survey (EMRS).

EMRS is an open-source repository for Extraction Method refactoring methodologies intended for supporting software engineering researchers and practitioners. For each case in EMRS, we provide summary, tools, dataset and original papers for extract method refactoring techniques. EMRS currently provides studies for 83 refactoring methods.

About

This survey collects and manages the relevant papers, datasets, and tools for utilization by researchers. This website was designed by Matheus Paixao, and Addl Tariq added content management as part of his MS Capstone at Rochester Institute of Technology. Dr. Mohamed Wiem Mkaouer and Dr. Eman Abdullah AlOmar contributed to the first incarnation of EMRS.

By leveraging the data contained in EMRS, software engineering researchers and practitioners can review the historical research on the Extract Method topic, and can utilize the existing studies (techniques, datasets, and tools) to improve or create new empirical studies to assess Extract Method Refactoring. Moreover, the data in EMRS is a valuable source of knowledge regarding motivation for and expansion of the Extract Method Refactoring techniques in the software engineering domain.

Here you can find useful information, such as understand the survey conducted. And download the tools, and datasets.

Download Tools

EMRS contains links to tools for multiple Extract refactoring techniques. All tools can be access through the below table containing links. The following table elaborates on the details concerning utilized artifacts by each project.

In the above table, you will find links to directories containing csv, jsonl and zip files for the tools, raw data and datasets. For more information on the utilization of tools for Extract Method Refactoring, please review the relevant papers Paper Reviews section.

Download Datasets

EMRS contains links to datasets for multiple Extract refactoring techniques. All tools can be access through the below table containing links.

The following table elaborates on the datasets details utilized for Long Method Decompositions by each project.

The following table elaborates on the datasets details utilized for Code Clone Extraction by each project.

The following table elaborates on the datasets details utilized for Separation of Concerns by each project.

In the above table, you will find links to directories/files containing csv, jsonl and zip files for datasets. For more information on the utilization of datasets for Extract Method Refactoring, please see the relevant papers in the Paper Reviews section.

Study Summary

EMRS contains a study for multiple Extract refactoring techniques. The literature study, experiments and table below summarizes the study details of this project.

Behind the Intent of Extract Method Refactoring: A Systematic Literature Review

Aim: In this paper, we aim to review the current body of knowledge on existing Extract Method refactoring research and explore their limitations and potential improvement opportunities for future research efforts. That is, the Extract Method is considered one of the most widely-used refactorings, but difficult to apply in practice as it involves low-level code changes such as statements, variables, parameters, return types, etc. Hence, researchers and practitioners begin to be aware of the state-of-the-art and identify new research opportunities in this context.

Method: We review the body of knowledge related to Extract Method refactoring in the form of a systematic literature review (SLR). After compiling an initial pool of 1,367 papers, we conducted a systematic selection, and our final pool included 83 primary studies. We define three sets of research questions and systematically develop and refine a classification schema based on several criteria including their methodology, applicability, as their degree of automation.

Results: The results construct a catalog of 83 Extract Method approaches indicating that several techniques have been proposed in the literature. Our results show that: (i) 38.6% of Extract Method refactoring studies primarily focus on addressing code clones; (ii) Several of the Extract Method tools incorporate the developer’s involvement in the decision-making process when applying the method extraction, and (iii) these existing benchmarks are heterogeneous and do not contain the same type of information, making standardizing them for the purpose of benchmarking difficult.

Conclusions: Our study serves as an “index” to the body of knowledge in this area for researchers and practitioners in determining the Extract Method refactoring approach that is most appropriate for their needs. Our findings also empower the community with information to guide future refactoring tool development.

We drive our study using the following research questions:
RQ1: What approaches were considered by the PSs to recommend Extract Method refactoring?
RQ2: What are the main characteristics of Extract Method recommendation tools?
RQ3: What are the datasets, and benchmarks used for evaluating and validating Extract Method recommendation tools?

Research Questions and Findings

RQ1. What approaches were considered by the PSs to recommend Extract Method refactoring?

Figure below of the sankey diagram provides detailed mappings between our three dimensions. Separation of Concerns is the most popular intent driving method extraction, followed up with Long Method, and finally Code Clone. Interestingly, this is not being matched in terms of the toolset, as the highest ratio of tools goes to Long Method with 41.2%, then equally between Code Clone and Separation of Concerns with 29.4%. Such observation has caught our attention particularly as Separation of Concerns is the only category that relies on all existing detection techniques and has its own unique one,i.e., Evolutionary-based, and yet, there is a lack of concretizing this amount of research into practical tools. As for Detection, it is no surprise that rule-based techniques are the most popular in identifying need-to-refactor code fragments. This is being inherited from how research couples refactoring to a natural response to code smells, e.g., Long Method.

Table analysis

Each research paper's approach methodology, type of data, decision metrics, and evaluation metrics are detailed below:

RQ2. What are the main characteristics of Extract Method recommendation tools?

To help select an appropriate Extract Method refactoring tool, we report the following main characteristics that can be considered to make an informed decision abouttools usage:

• Language: Indicates the programming language the tool supports.
• Intent: Indicates the context in which the tool can be used.
• Number of Metrics: Indicates the number of software metrics used by the tool.
• Interface: Indicates what IDE the tool supports.
• Usage Guide?: Indicates the availability of instructions on how to use the tool.
• Tool Link: Indicates to the online source code repository.
• Last Update: Indicates whether the tool has been consistently updated/maintained since its development

RQ3. What are the datasets, and benchmarks used for evaluating and validating Extract Method recommendation tools?

Out of the 83 primary studies analyzed, almost 78% of the datasets are unavailable to the public, with only 22% available online, which means there is a lack of online datasets for the Extract Method refactoring research. Primary studies have mostly employed small or medium-scale open-source applications, often developed using Java, typically containing less than 225,000 lines of code. These dataset are heterogeneous and do not contain the same type of information, making standardizing them for the purpose of benchmarking difficult.

The detailed information on Tools, Datasets and Original papers can be found in the following sections:

Paper Reviews

The following list elaborates on the papers surveyed for this study (Paper Links are clickable).

Data Collection

• ScienceDirect
• Scopus
• Springer Link
• Web of Science
• ACM Digital Library
• IEEE Xplore
• Wiely

Final Set

• Final set of articles (Summary)

Venues

• Publication Venues

Manual Classification

• Primary Studies Classification

Publications

This is a list of publications that use the CROP dataset. If you have a published piece of work that uses CROP and it is not listed above, feel free to contact us. Your publication will be included in the list soon.

Journal Articles

Matheus Paixao, Jens Krinke, DongGyun Han, Chaiyong Ragkhitwetsagul, Mark Harman. 2019. In IEEE Transactions on Software Engineering (TSE). Preprint

Conference Papers

Luca Pascarella, Davide Spadini, Fabio Palomba, Alberto Bacchelli. 2020. On The Effect Of Code Review On Code Smells. In IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). Preprint

Matheus Paixao, Paulo Henrique Maia. 2019. Rebasing in Code Review Considered Harmful:A Large-scale Empirical Investigation. In International Conference on Source Code Analysis and Manipulation (SCAM). Preprint

Matheus Paixao, Jens Krinke, DongGyun Han, and Mark Harman. 2018. CROP: Linking Code Reviews to Source Code Changes. In International Conference on Mining Software Repositories (MSR). Preprint

Contact

You can contact the EMRS's team through the following channel:

EMRS's mailing list