Ok let’s start to understand the rewrite engine.
Let’s do it with an example, our goal:Find all the methods that send the consecutives mesages: #globlals #at:
In our example we want as result:
findClassesForCategories: aCollection | items | aCollection isEmpty ifTrue: [ ^ self baseClass withAllSubclasses asSet ]. items := aCollection gather: [ :category | ((Smalltalk organization listAtCategoryNamed: category) collect: [ :each | Smalltalk globals at: each ]) select: [ :each | each includesBehavior: self baseClass ] ]. ^ items asSet
But not with this variation:
findClassesForCategories: aCollection | items | aCollection isEmpty ifTrue: [ ^ self baseClass withAllSubclasses asSet ]. items := aCollection gather: [ :category | ((Smalltalk organization listAtCategoryNamed: category) collect: [ :each | Smalltalk globals at: each ifAbsent: [ "ignore" ]]) select: [ :each | each includesBehavior: self baseClass ] ]. ^ items asSet
Why? because in the second example we send the messages #globals (ok we want this!) but then we send #at:ifAbsent: (we don’t want this)
Ok so… Which object do I send wich message to find my “problematics” methods?
The answer is very simple a RULE… For each rule you should extend one flavor of rule and define what you want to do. Sounds easy let’s check the rules hierarchy
All of those are abstract classes, so we will take a look into them and choose the one that is better for our case.
- RBLintRule: defines the protocol used for execute a rule.
This is a very general class, doesn’t define:
– what to do when we are cheeking a class, for this we have to implement: #checkClass:. In our case we don’t care about the class, so we can let the default that does nothing with the class
– what to do when we are cheeking methods, for this we have to implement: #checkMethod:.
In our case we should obtain the source code form the method and search in the source code all the variants for the consecutive messages #globals #at:, an inocent version of this can be:
A way to do it:
checkMethod: aContext (aContext compiledMethod source findString: 'globals at:') ~= 0 ifTrue:["we have a match in aContext selectedClass>> aContext selector we should handle it"]
I said that it is an innocent implementation because if we have a method with the code: “(…) globals “yes a comment here!” at: (…)” or “(…) globals at: number” we aren’t causing a match and if we have: “(…) globals at: something ifAbsent: (…)” we are and we shouldn’t.
Probably we should allways use one subclass of LintRule with more features implemented otherwise we have to extend LintRule to define what to do in case of matching, model a result, transforming code (…)
- RBBasicLintRule: it’s another abstract class that adds a result associated to a rule. We still should redefine checkMethod and/or check class but we have a proposal for a result:
checkMethod: aContext (aContext compiledMethod source findString: 'globals at:') ~= 0 ifTrue:[result addClass: aContext selectedClass selector: aContext selector ]
This can be a good extension point, but again… you have a lot to implement here.
- RBBlockLintRule: it’s an abstract class, that by default specifies that the resultClass corresponds to a selector environment.
This class doesn’t add a lot of behavior but we reduce errors related to the result handling.
- RBParseTreeLintRule: here is where everything gets interesting.
The main point in this class is to offer an implementation for #checkMethod: the implementation motivation is to check methods using ast’s representations.
We will have a wanted tree and then we will obtain the AST for the checked method, in order to find a match we observe if the wanted tree is subtree of the method tree.
Also it’s posible to define metavariables where we can specify that we want to match with a node type but not necesary the value of it.
In our example our matching expression will look like:
“@lotOfStuffBefore globals at: “@lotOfStuffAfter
With this expresion we are saying:
– “@lotOfStuffBefore: ignore everything before globals (receiver / message send):
– “@lotOfStuffAfter: ignore everything after at: (object / messages)
(For more information about metavariables see my first post)
The solution implemented for the check method introduces new objects:
– Matcher: Is the responsable for visit an AST and verify the matches. Is the matcher the one who interprets the matching expressions. First of all we have to specify if we are matching with a method (matchesMethod: aString do: aBlock) or an expreesion (matches: aString do: aBlock) pattern.
We can add more than one matching expression, it’s very important to know that every time we add an expression in the matcher inside we are adding a RBSearchRule for that expression.
When the tree rule sends the message executeTree: to the matcher at the end the matchers iterates over all the rules visiting the nodes delegating in the RBSearchRules to perform the match.
It’s important to say that the search rules aren’t deleted automatically, so, if you want to reuse the object probably you should reset the rule, reseting the matcher.
– ParseTreeEnvironment: Is an specialized SelectorEnvironment that makes it possible to detect the selection interval for an expression inside the method, using the ast matcher
This class is still abstract because we should categorize it, adding a name and initilize the rule with the matching expression patterns.
In functionality is almost the same that before but we have other abstraction level, the result is handled automatically, and we don’t have to worry about checking a method or a class only to define the desired matching expression.
- RBCompositeLintRule: Is just a composite for rules.
- RBTransformationRule: The main idea here is to produce a transformation in the system, for this implements: #checkMethod: in a similar way that RBParseTreeLintRule the diference is that if we find a match we will produce a modification in the code and then we change the method with the new version of the code (recompiling the new method).
In order to solve this the rule adds some objects:
– RBParseTreeEnvironment: with all the results, the results are: RBAddMethodChange to track a change in a method
– RBParseTreeRewriter: It’s a subclass off the matcher (RBParseTreeSearcher) and again the main point is that this is a visitor that works over a method AST changing it depending in the matching and transforming expression.
The transforming expression also works with metavariable and usually we use the metavariables defined in the matching expression to specify the transformation.
- Before start you should choose if you want to:
perform a search
do a match
- Implement your rules because all of them are abstract, probably you will end up using the Tree rules because are more automatics and powerfull than to basics.
- When you are defining your rule:
– give a name for it
– define if you will use a method or an expression type pattern
– write your patterns and add them to the rule
– if you are in a search rule to the matcher
– if you are in a transformation rule rewriteRule
– define what to do with a result
- run your rule
- use your results
- if you want to reuse it reset your rule, again if it’s a search rule, reset the matcher, if it’s a transformation rule the rewriteRule.
In our example:
- we want to match, so let’s create an object that extends: RBParseTreeLintRule:
RBParseTreeLintRule subclass: #SearchGlobalsAtUsage instanceVariableNames: '' classVariableNames: '' poolDictionaries: '' category: 'Blog-example'
- we have to implement the abstract methods:
name ^ 'Find all potential wrong usage in with globals'
- I want to match an expression type because I do not care about the rest of the method, I want everything that contains the messages: #globals #at:, we also have to say what to do with the matching node in this example I will open an inspector:
initialize super initialize. self matcher matches: '``@lotOfStuffBefore globals at: ``@lotOfStuffAfter' do: [:theMatch :theOwner | theMatch inspect].
- now we should run the rule:
WARNING: this can take a time because you will check the hole system
SearchGlobalsAtUsage new run.
To avoid this you can restrict the environment for your rule, an example:
rule := SearchGlobalsAtUsage new. environment := RBClassEnvironment class: Result. RBSmalllintChecker runRule: rule onEnvironment: (environment).
If you have matches then you will see the inspector.
So, as we can see this is quite complex, and in the sinposis you can see that before doing anything you have to make too many decisions, the idea behind Flamel is to make it easier.
The equivalent code (with a restricted environment) using Flamel for all this is:
FlamelMatchAndTransformRule new matchingExpression: '``@lotOfStuffBefore globals at: ``@lotOfStuffAfter'; scope: environment; run; result
If you evaluate that and inspect it you can search your results 🙂
I think this is quite cool to replace all that code (with class creation included) with 5 lines.
And this was all for understanding a little bit the rewrite engine and see Flamel in action for today