NDepend Query to show const, enum, struct dependencies - ndepend

I wrote the following query in order to determine the dependencies between my solution and other assemblies. We have a large library of Internal Nuget shared libraries which is used extensively, and I want to ensure these are included - hence I'm using 't' below to eliminate certain 3rd party assemblies but including our internal libraries.
This query works great, but I have realised that it only shows us dependencies where the dependency is a method call. It doesn't include Constants, enums and structs.
How can I enhance the query below to show us the detail of these and any other dependencies?
let t = Assemblies.WithNameWildcardMatchIn("xunit","RestSharp","NSubstitute","O2S*","EntityFramework","AxInterop*","AutoMapper","Autofac*","ADODB","mscorlib","System*", "Microsoft*","Infra*","Interop*").ToDictionary<IAssembly,String>(c=>c.Name)
from a in Application.Assemblies
from m in a.ChildMethods
from b in m.MethodsCalled
let isThirdParty = t.ContainsKey(b.ParentAssembly.Name)
select new { a,isThirdParty,m.ParentNamespace, m.ParentType,m.Name,DependsOnAssembly=b.ParentAssembly.Name, DependsOnNamespace=b.ParentNamespace,DependsOnParentType=b.ParentType,DependsOnMethod=b.Name}

What about this refactored version of your query:
from a in Application.Assemblies
from m in a.ChildMethods
from b in m.MethodsCalled.Cast<IMember>().Union(m.FieldsUsed.Cast<IMember>())
let isThirdParty = b.IsThirdParty
select new {
a,
isThirdParty,
m.ParentNamespace,
m.ParentType,
m.Name,
DependsOnAssembly=b.ParentAssembly.Name,
DependsOnNamespace=b.ParentNamespace,
DependsOnParentType=b.ParentType,
DependsOnMember=b.Name
}
First we simplified it greatly by using b.IsThirdParty :)
Second we do a Union<IMember>() between MethodsCalled and FieldsUsed. So you get also all fields read and/or assigned in addition to methods called.
Concerning structure usage, as long as you use a member of the structure (constructor, property, field...) the dependency will be listed.
Concerning enum, if a method uses an enumeration, you'll see a dependency toward the instance field EnumName.value__.
However you won't see usage of constant nor enumeration values. The reason is that this information get lost in the IL code that NDepend analyze. Constant (and enumeration values are also constants) are replaced with their values within the IL code.
Hope this help !
As a side note, isn't the query result more readable from within the NDepend UI if you write instead:
from m in Application.Methods
select new {
m,
thirdPartyMethodsCalled = m.MethodsCalled.Where(m1 => m1.IsThirdParty),
applicationMethodsCalled = m.MethodsCalled.Where(m1 => !m1.IsThirdParty),
thirdPartyFieldsUsed = m.FieldsUsed.Where(m1 => m1.IsThirdParty),
applicationFieldsUsed = m.FieldsUsed.Where(m1 => m1.IsThirdParty)
}

Related

Pass information from one compiler component to another without mutation

I am building a compiler plugin that has two components
Permission Accumulator: Load the function definitions and some extra meta data about them into a structure like a Map[String, (...)] where String keys represents the function name and the tuple contains the meta information + the definition in scope.
Function Transformer: Recursively traverse the function bodies to check if the metadata of the caller aligns with the callee. More specifically caller.metadata ⊆ callee.metada
This kind of preloading is a rather common thing in compilers (Zinc, Unison etc. all have similar tricks they pull). The first component needs to pass this information it has accumulated to the second component.
Unfortunately the current implementation uses a mutable.Map in the Plugin class and initiates the phases with a reference to this mutable Map. While given the fact that this code won't be surfaced to the end user and some amount of mutation could be tolerated, if someone (including myself) were to add another component/phase that touched this Map in the future, things can go very wrong, resulting in a situation that is painful to debug.
Question: I am wondering if there is a way to instantiate one component, extract some information from it, use that info to init the second component and run it.
Current Implementation:
import scala.collection.mutable.{ Map => MMap }
class Contentable(override val global: Global) extends Plugin {
val functions: MMap[String, (String, List[String])] = MMap()
val components = new PermissionAccumulator(global, functions) :: new FunctionRewriter(global, functions) :: Nil
}
The first component mutates the Map as such:
functions += (dd.name.toString -> ((md5HashString(dd.rhs.toString()), roles)))
What I have tried:
Original plan was to encapsulate the mutation inside the first component and do something like secondComponent(global, firstComponent.functions) but because Scala class create a copy of their arguments when an instance is created, the changes to this Map is not reflected in the second component
Note: I have no problem turning these component to phases if that makes a difference.

Losing path dependent type when extracting value from Try in scala

I'm working with scalax to generate a graph of my Spark operationS. So, I have a custom library that generates my graph. So, let me show a sample:
val DAGWithoutGet = createGraphFromOps(ops)
val DAGWithGet = createGraphFromOps(ops).get
The return type of DAGWithoutGet is
scala.util.Try[scalax.collection.Graph[typeA, scalax.collection.GraphEdge.DiEdge]],
and, for DAGWithGet is
scalax.collection.Graph[typeA, scalax.collection.GraphEdge.DiEdge].
Here, typeA is a project related class representing a single Spark operation, not relevant for the context of this question. (for context only: What my custom library does is, essentially, generate a map of dependencies between those operations, creating a big Map object, and calling Graph(myBigMap: _*) to generate the graph).
As far as I know, calling the .get command on this point of my code or later should not make any difference, but that is not what I'm seeing.
Calling DAGWithoutGet.get.nodes has a return type of scalax.collection.Graph[typeA,DiEdge]#NodeSetT,
while calling DAGWithGet.nodes returns DAGWithGet.NodeSetT.
When I extract one of those nodes (using the .find method), I receive scalax.collection.Graph[typeA,DiEdge]#NodeT and DAGWithGet.NodeT types, respectively. Much to my dismay, even the methods available in each case are different - I cannot use pathTo (which happens to be what I want) or withSubgraph on the former, only on the latter.
My doubt is, then, after this relatively complex example: what is going on here? Why extracting the value from the Try construct on different moments leads to different types, one path dependent, and the other not - or, if that isn't correct, what may I be missing here?

Documentation comment for loop variable in Xcode

I know that we can use
/// index variable
var i = 0
as a documentation comment for a single variable.
How can we do the same for a loop variable?
The following does not work:
var array = [0]
/// index variable
for i in array.indices {
// ...
}
or
var array = [0]
for /** index variable */ i in array.indices {
// ...
}
Background:
The reason why I don’t use "good" variable names is that I’m implementing a numerical algorithm which is derived using mathematical notation. It has in this case only single letter variable names. In order to better see the connection between the derivation and the implementation I use the same variable names.
Now I want to comment on the variables in code.
The use of /// is primarily intended for use of documenting the API of of a class, struct, etc. in Swift.
So if used before a class, func, a var/let in a class/struct, etc. you are attaching documentation to that code aspect that Xcode understands how to show inline. It doesn’t know how to pickup that information for things inside of function since at this time that is not the intention of /// (it may work for simple var/let but not likely fully on purpose).
Instead use a simple // code comment for the benefit of any those working in the code however avoid over documenting the code since good code is likely fairly self explaining to anyone versed in the language and adding unneeded documentations can get in the way of just reading the code.
This is a good reference for code documentation in Swift at this time Swift Documentation
I woud strongly push back on something like this if I saw it in a PR. i is a massively well adopted "term of art" for loop indices. Generally, if your variable declaration name needs to be commented, you need a better variable name. There are some exceptions, such as when it stores data with complicated uses/invariants that can't be captured in a better way in a type system.
I think commenting is one area that beginners get wrong, mainly from being misled by teachers or by not yet fully understanding the purpose of comments. Comments don't exist to create an english based, psuedo-programming language in which your entire app will be duplicated. Understanding the programming language is a minimal expectation out of contributors to a project. Absolutely no comments should be explaining programming language features. E.g. var x: Int = 0 // declares a new mutable variable called x, to the Int value 0, with the exception of tutorials for learning Swift.
Commenting in this manner might seem like it's helpful, because you could argue it explains things for beginners. That may be the case, but it's suffocating for all other readers. Imagine if novel had to define all the English words they used.
Instead, the goal of documentation to explain the purpose and the use of things. To answer such questions as:
Why did you implement something this way, and not another way?
What purpose does this method serve?
When will this method of my delegate be called?
Case Study: Equatable
For a good example, take a look at the documentation of Equatable
Some things to notice:
It's written for an audience of Swift developers. It uses many things, which it does not explain such as, arrays, strings, constants, variable declaration, assignment, if statements, method calls (such as Array.contains(_:)), string interpolation, the print function.
It explains the general purpose of this protocol.
It explains how to use this protocol
It explains how you can adopt this protocol for your own use
It documents contractual requirements that cannot be enforced by the type system.
Since equality between instances of Equatable types is an equivalence relation, any of your custom types that conform to Equatable must satisfy three conditions, for any values a, b, and c:
a == a is always true (Reflexivity)
a == b implies b == a (Symmetry)
a == b and b == c implies a == c (Transitivity)
It explains possible misconceptions about the protocol ("Equality is Separate From Identity")

Spring Data Neo4j - ORDER BY {order} fails

I have a query where the result should be ordered depending on the passed parameter:
#Query("""MATCH (u:User {userId:{uid}})-[:KNOWS]-(:User)-[h:HAS_STUFF]->(s:Stuff)
WITH s, count(h) as count ORDER BY count {order}
RETURN o, count SKIP {skip} LIMIT {limit}""")
fun findFromOthersByUserIdAndSortByAmountOfStuff(
#Param("uid") userId: String,
#Param("skip") skip: Int,
#Param("limit") limit: Int,
#Param("order) order: String): List<StuffWithCountResult>
For the order parameter I use the following enum and its sole method:
enum class SortOrder {
ASC,
DESC;
fun toNeo4JSortOrder(): String {
when(this) {
ASC -> return ""
DESC -> return "DESC"
}
}
}
It seems that SDN does not handle the {order} parameter properly? On execution, I get an exception telling that
Caused by: org.neo4j.kernel.impl.query.QueryExecutionKernelException: Invalid input 'R': expected whitespace, comment or a relationship pattern (line 3, column 5 (offset: 244))
" RETURN o, count SKIP {skip} LIMIT {limit}"
^
If I remove the parameter from the Cypher statement or replace it with a hardcoded DESC the method succeeds. I believe it's not because of the enum since I use (other) enums in other repository methods and all these methods succeed. I already tried a different parameter naming like sortOrder, but this did not help.
What am I missing here?
This is the wrong model for changing sorting and paging information. You can skip to the answer below for using those options, or continue reading for an explanation of what is wrong in your code as it stands.
You cannot bind where things aren't allowed to be bound:
You cannot bind a parameter into a syntax element of the query that is not setup for "parameter binding". Parameter binding doesn't do simple string substitutions (because you would be open for injection attacks) but rather uses binding API's to bind parameters. You are treating the query annotation like it is performing string substitution instead, and that is not what is happening.
The parameter binding docs for Neo4J and the Java manual for Query Parameters show exactly where you can bind, the only places allowed are:
in place of String Literals
in place of Regular Expressions
String Pattern Matching
Create node with properties, as the properties
Create multiple nodes with properties, as the properties
Setting all properties of a node
numeric values for SKIP and LIMIT
as the Node ID
as multiple Node IDs
Index Value
Index Query
There is nothing that says what you are trying is allowed, binding in the ORDER BY clause.
That isn't to say that the authors of Spring Data couldn't work around this and allow binding in other places, but it doesn't appear they have done more than what Neo4J Java API allows.
You can instead use the Sort class:
(the fix to allow this is marked for version 4.2.0.M1 which is a pre-release as of Sept 8, 2016, see below for using milestone builds)
Spring Data has a Sort class, if your #Query annotated method has a parameter of this type, it should apply sorting and allow that to dynamically modify the query.
I assume the code would look something like (untested):
#Query("MATCH (movie:Movie {title={0}})<-[:ACTS_IN]-(actor) RETURN actor")
List<Actor> getActorsThatActInMovieFromTitle(String movieTitle, Sort sort);
Or you can use the PageRequest class / Pageable interface:
(the fix to allow this is marked for version 4.2.0.M1 which is a pre-release as of Sept 8, 2016, see below for using milestone builds)
In current Spring Data + Neo4j docs you see examples using paging:
#Query("MATCH (movie:Movie {title={0}})<-[:ACTS_IN]-(actor) RETURN actor")
Page<Actor> getActorsThatActInMovieFromTitle(String movieTitle, PageRequest page);
(sample from Cypher Examples in the Spring Data + Neo4j docs)
And this PageRequest class also allows sorting parameterization. Anything that implements Pageable will do the same. Using Pageable instead is probably more proper:
#Query("MATCH (movie:Movie {title={0}})<-[:ACTS_IN]-(actor) RETURN actor")
Page<Actor> getActorsThatActInMovieFromTitle(String movieTitle, Pageable page);
You might be able to use SpEL in earlier versions:
As an alternative, you can look at using SpEL expressions to do substitutions in other areas of the query. I am not familiar with it but it says:
Since this mechanism exposes special parameter types like Sort or Pageable as well, we’re now able to use pagination in native queries.
But the official docs seem to say it is more limited.
And you should know this other information:
Here is someone reporting your exact same problem in a GitHub issue. Which then leads to DATAGRAPH-653 issue which was marked as fixed in version 4.2.0.M1. This references other SO questions here which are outdated so you should ignore those like Paging and sorting in Spring Data Neo4j 4 which are no longer correct.
Finding Spring Data Neo4j Milestone Builds:
You can view the dependencies information for any release on the project page. And for the 4.2.0.M1 build the information for Gradle (you can infer Maven) is:
dependencies {
compile 'org.springframework.data:spring-data-neo4j:4.2.0.M1'
}
repositories {
maven {
url 'https://repo.spring.io/libs-milestone'
}
}
Any newer final release should be used instead.

How do purely functional compilers annotate the AST with type info?

In the syntax analysis phase, an imperative compiler can build an AST out of nodes that already contain a type field that is set to null during construction, and then later, in the semantic analysis phase, fill in the types by assigning the declared/inferred types into the type fields.
How do purely functional languages handle this, where you do not have the luxury of assignment? Is the type-less AST mapped to a different kind of type-enriched AST? Does that mean I need to define two types per AST node, one for the syntax phase, and one for the semantic phase?
Are there purely functional programming tricks that help the compiler writer with this problem?
I usually rewrite a source (or an already several steps lowered) AST into a new form, replacing each expression node with a pair (tag, expression).
Tags are unique numbers or symbols which are then used by the next pass which derives type equations from the AST. E.g., a + b will yield something like { numeric(Tag_a). numeric(Tag_b). equals(Tag_a, Tag_b). equals(Tag_e, Tag_a).}.
Then types equations are solved (e.g., by simply running them as a Prolog program), and, if successful, all the tags (which are variables in this program) are now bound to concrete types, and if not, they're left as type parameters.
In a next step, our previous AST is rewritten again, this time replacing tags with all the inferred type information.
The whole process is a sequence of pure rewrites, no need to replace anything in your AST destructively. A typical compilation pipeline may take a couple of dozens of rewrites, some of them changing the AST datatype.
There are several options to model this. You may use the same kind of nullable data fields as in your imperative case:
data Exp = Var Name (Maybe Type) | ...
parse :: String -> Maybe Exp -- types are Nothings here
typeCheck :: Exp -> Maybe Exp -- turns Nothings into Justs
or even, using a more precise type
data Exp ty = Var Name ty | ...
parse :: String -> Maybe (Exp ())
typeCheck :: Exp () -> Maybe (Exp Type)
I cant speak for how it is supposed to be done, but I did do this in F# for a C# compiler here
The approach was basically - build an AST from the source, leaving things like type information unconstrained - So AST.fs basically is the AST which strings for the type names, function names, etc.
As the AST starts to be compiled to (in this case) .NET IL, we end up with more type information (we create the types in the source - lets call these type-stubs). This then gives us the information needed to created method-stubs (the code may have signatures that include type-stubs as well as built in types). From here we now have enough type information to resolve any of the type names, or method signatures in the code.
I store that in the file TypedAST.fs. I do this in a single pass, however the approach may be naive.
Now we have a fully typed AST you could then do things like compile it, fully analyze it, or whatever you like with it.
So in answer to the question "Does that mean I need to define two types per AST node, one for the syntax phase, and one for the semantic phase?", I cant say definitively that this is the case, but it is certainly what I did, and it appears to be what MS have done with Roslyn (although they have essentially decorated the original tree with type info IIRC)
"Are there purely functional programming tricks that help the compiler writer with this problem?"
Given the ASTs are essentially mirrored in my case, it would be possible to make it generic and transform the tree, but the code may end up (more) horrendous.
i.e.
type 'type AST;
| MethodInvoke of 'type * Name * 'type list
| ....
Like in the case when dealing with relational databases, in functional programming it is often a good idea not to put everything in a single data structure.
In particular, there may not be a data structure that is "the AST".
Most probably, there will be data structures that represent parsed expressions. One possible way to deal with type information is to assign a unique identifier (like an integer) to each node of the tree already during parsing and have some suitable data structure (like a hash map) that associates those node-ids with types. The job of the type inference pass, then, would be just to create this map.