ASTGen: Automated Testing of Refactoring Engines

Welcome to the ASTGen Project home page. ASTGen is an imperative, iterative, composable and generic test data generation framework used to create abstract syntax trees. It can be used to test many applications that take programs as input. We have applied it to testing the Eclipse and NetBeans refactoring engines.

Publications

Automated Testing of Refactoring Engines

Brett Daniel, Danny Dig, Kely Garcia, Darko Marinov.
In ESEC/FSE 2007: 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering
Dubrovnik, Croatia, September 2007.

• PDF
• PS
• BibTex
• Presentation

Reducing the costs of bounded-exhaustive testing

Vilas Jagannath, Yun Young Lee, Brett Daniel, Darko Marinov.
In FASE 2009: Fundamental Approaches to Software Engineering
York, UK, March 2009.

• PDF
• BibTex
• Presentation

Downloads

The following table contains source and binary downloads for the ASTGen projects. Compilation instructions and licenses are bundled with the download.

Supplemental Information

Results

The following links show the reports for all bugs that ASTGen found.

Eclipse Bugs

• 177093: [encapsulate field] Encapsulating parenthesized field with unary operator yields compilation error
• 177095: [encapsulate field] Encapsulating parenthesized field assignment yields compilation error
• 177099: [encapsulate field] Encapsulate field on an array type does not use correct type for accessors
• 177182: [encapsulate field] Encapsulate field on byte with unary operator yields compilation error
• 177636: [push down] Push down field results in compilation error
• 177098: [encapsulate field] Attempting to encapsulate a field assigned to itself does not encapsulate the field read
• 177195: [encapsulate field] Encapsulating a field referenced by a super call yields a compilation error
• 177944: [move member type] Super field reference to enclosing type results in compilation error
• 178085: [push down] Push down field to class declared locally is disallowed
• 186252: [push down] Push Down a referenced field yields compilation error
• 186253: [encapsulate field] Encapsulate a referenced field changes behavior
• 194996: [encapsulate field] Encap. a field yields comp. error due to override getter
• 194997: [encapsulate field] Encap. field yields comp. error due to incompatible return type
• 194998: [encapsulate field] Encap. field yields comp. error due to field not visible
• 195000: [pull up] field that references another field declared in its superclass yields comp. error
• 195001: [pull up] Pull up field declared in an internal class that references another field yields comp.error
• 195002: [pull up] Pull up a field yields comp. error due to field not visible
• 195003: [push down] compile error with 'this' qualifier
• 195004: [push down] error due to undefined method
• 195005: [pull up] yields comp. error due to no enclosing instance
• 195006: [pull up] PullUp method yields comp. error due to not visible field

NetBeans Bugs

• 98054: [encapsulate field] Encapsulating parenthesized field with unary operator yields compilation error
• 98055: [encapsulate field] Encapsulating parenthesized field assignment is not performed correctly
• 98056: [encapsulate field] Encapsulate field on byte with unary operator yields compilation error
• 98057: [encapsulate field] Encapsulating a field referenced by a super call is not performed correctly
• 98058: [rename field] Renaming parenthesized field assignment yields compilation error
• 98165: [Encapsulate Field] Refactoring does not distinguish between assignment and evaluation
• 98166: [Push Down Field] Refactoring does not replace field reference
• 98167: [Push Down Field] Refactoring is forbidden even though it should be allowed
• 98847: [Move Inner To Outer Level] Move Inner Class B yields compilation error
• 98848: [Move Inner To Outer Level] Move Inner Class B which has an expression that involves field A.f yields compilation error
• 103591: [Push Down Field] Push Down a referenced field yields compilation error
• 103592: [Push Down Field] Push down field should return a warning
• 108473: [encapsulate field] Encap. field that has a getter yields comp. error
• 108474: [encapsulate field] Encap. field results in change of behavior
• 108477: [pull up field] PullUp a field initialized with super.f yields comp. error
• 108478: [pull up field] PullUp a field initialized with A.this.f yields comp. error
• 108479: [pull up field] PullUp a field initialized with new A().f yields comp. error
• 108480: [push down field] PushDown field returns an erroneous message (DUPLICATED)
• 108481: [push down field] PushDown a field referenced in another class returns an error message (DUPLICATED)
• 108482: [push down method] PushDown a method that references a field
• 108484: [push down method] PushDown a method referenced in another class returns an error message
• 108485: [push down method] PushDown a method referenced inside the same class where it is declared returns an erroneous message
• 108486: [pull up method] PullUp method that references a field yields comp. error
• 108487: [pull up method] PullUp method that is referenced by other method yields comp. error
• 108488: [pull up method] PullUp a method that references a private field yields comp. error
• 108489: [encapsulate field] Encap. field yields comp. error due to incompatible return type

Generator List

The following list describes each of the complex generators used in the paper.

FieldReference

Generates many classes. Each class contains a field and a method that references the field in some way.

ClassRelationship

Generates pairs of classes that are related by inheritance, reference, or containment.

DoubleClassFieldReference

Generates pairs of classes related in some way. One class declares a field and the other references it in some way.

MethodReference

Generates many classes with two methods. One method calls the other and may overload it.

MethodParamReference

Generates method declarations, each of which has a parameter referenced.

ClassArrayField

Generates classes, each of which declares a field of some array type.

SingleClassTwoFields

Generates classes, each of which declares two fields. One field references the other in its initialization.

DoubleClassGetterSetter

Generates pairs of classes in an inheritance relationship. The superclass declares a field; either class declares a getter or setter for the field. The superclass declares a main method that references the getter or setter.

DoubleClassParentField

Generates pairs of classes in an inheritance relationship. The superclass declares a field. Either class declares another field that references the first in its initializer.

DoubleClassParentMethod

Generates pairs of classes in an inheritance relationship. The superclass declares a field; the subclass declares a method that optionally references the field. Either class declares a second method that references the first method.

DoubleClassChildField

Generates pairs of classes in a inheritance relationship. The superclass declares a field, and the subclass declares another field that optionally references the first in its initializer.

DoubleClassChildMethod

Generates pairs of classes in a inheritance relationship. The superclass declares a field and the subclass declares two methods. One method optionally references the field and the second method references the first method.

TripleClassChildField

Generates triplets of classes such that two of them have an inheritance relationship. The subclass declares a field that references another field declared in either of the other classes.

TripleClassChildMethod

Generates triples of classes such that two of them have a inheritance relationship. The subclass declares two methods. The first method references a field declared in one of the other two classes and the second method references the first method.

Experimental Data

This directory tree contains our raw experimental data. The tree has the following structure:

/<refactoring>/<generator>.zip/<generator>/test<#>/(in|out|outNB)/

The "in" directory contains the generated input file(s); "out" contains the result of the refactoring in Eclipse; and "outNB" contains the result of the refactoring in NetBeans (when applicable).

In addition, an "out" directory contains one of the following result files. Note that "outNB" does not contain analogous files because we used the reports built in to NetBeans' testing framework.