Sparx Systems Forum

Enterprise Architect => Bugs and Issues => Topic started by: dschmid2 on July 07, 2010, 10:57:15 pm

Title: Bug in GetAllLatest
Post by: dschmid2 on July 07, 2010, 10:57:15 pm
We discovered a serious bug concerning the retrieval of packages from SourceControl.
When doing a GetAllLatest in a model, I want that all package XML files are retrieved from SourceControl and (forced) imported into my model. This should replace all model elements with the ones that are read from the files.
When I randomly choose a package afterwards and do a "Compare with Controlled Version", there are differences showing up. I can do as many GetAllLatest as I want, it doesn't change.
Only when I do a GetLatest of the particular package, the package is finally imported and the content replaced. An subsequent "Compare" shows that no more differences are detected.

This bug must have been introduced in one of the latest releases...

The GetAllLatest functionality is not very stable anyway. We used to call it twice always to make sure that all packages and dependencies have been resolved.  :(
Title: Re: Bug in GetAllLatest
Post by: Geert Bellekens on July 07, 2010, 11:12:17 pm
Oh, serious indeed :o

Did you test on the latest version (8.0.858)?
Did you report it to sparx using the link (http://www.sparxsystems.com/support/bug_report.html) on the bottom of the page?

Geert
Title: Re: Bug in GetAllLatest
Post by: dschmid2 on July 07, 2010, 11:33:49 pm
Yes, we have 858 installed and I reported it to Sparx.
Title: Re: Bug in GetAllLatest
Post by: HowardB on July 13, 2010, 10:51:39 am
Daniel,

there has been no bug introduced into the Get All Latest functionality as you are suggesting.

Indeed, the basic principle of the Get All Latest functionality - that it recursively calls Get Latest, for each version controlled package in the model - has not changed since its inception.

That's right, Get All Latest simply calls Get Latest for each package, one after the other.

Now, when EA imports a package from XMI, it must first delete the existing package and its content from the model, before it can re-import it from the XMI.

When the package and its contents are deleted, that can have an effect on other packages in the model.

All associations using elements that are deleted will also be deleted.  
The elements that are deleted will also be removed from all diagrams on which they were previously shown.

EA simply cannot use elements that do not exist in the model.  

When the elements are re-imported into the model, there are certain situations in which the previously existing associations can not be restored.  Similarly there are some situations in which elements will not be replaced on the diagrams on which they previosuly appeared.

The order in which packages are re-imported into the model can have an effect on whether cross-package dependencies are correctly restored.

The Sparx Systems white paper on "Version Control Best Practices" details situations in which Cross Package Dependencies can be affected by XMI export/import, and gives some guidance on how to avoid and minimize the impact of these effects.

Daniel, you apparently have some understanding of the need to have all elements present in the model in order to resolve dependencies, but perhaps you didn't realise that forcing EA to re-import packages actually causes them to be deleted from the model before being re-imported.

Please DO NOT select the option to "force reload from XMI".

Those packages where new content has been committed to version control will be re-imported as necessary.

I believe this on its own will virtually eliminate the problems you are seeing.


We are currently working on some new functionality that will scan and reconcile XMI file content with the model DB, to "put-back" connectors and elements that are contained in the XMI, but might have been erroneously removed from the model, due to the order of re-importing packages.

Until then, please follow the guidelines in the Best Practices document.

I hope this helps,


Howard Britten.
Title: Re: Bug in GetAllLatest
Post by: Frank Horn on July 14, 2010, 06:22:54 am
Quote
When the elements are re-imported into the model, there are certain situations in which the previously existing associations can not be restored.  Similarly there are some situations in which elements will not be replaced on the diagrams on which they previosuly appeared.

The order in which packages are re-imported into the model can have an effect on whether cross-package dependencies are correctly restored.

So never trust EA to restore your data!

I wouldn't touch the "Get All Latest" command with a ten foot pole, except in one special situation:

When you build up a new project of controlled packages which contain nested controlled packages, import the root packages via the "Get Package" command, and then perform "Get All Latest" to fill the stubs.

In my experience, EA's version control is only safe if you stick to some very restrictive rules:

1. No cross references
Organize your controlled packages with acyclic dependencies (i.e if something in package1 has a depency on anything in package2, nothing in package2 must have a dependency on anything in package1, or anything depending on anything in package1).

2. One or all checkout
Only check out one package at a time unless you are aware of the dependency graph. If you are aware: when changing packages on which other packages depend, check out ALL dependent packages as well.

3. Communicate
After checking in, inform all users of the changed packages and urge them to update their projects.

4. Update before check out
Before you check out a package depending on other packages, perform "Get Latest" on all packages on which it depends. Don't believe EA when it tells you a package is up to date, but ENFORCE reloading from XMI!

5. Update depth first
When performing "Get Latest", do it individually on each package, and do it in order of dependency (i.e first the packages which depend on nothing else, then those which only depend on the packages previously updated, and so on).
Title: Re: Bug in GetAllLatest
Post by: HowardB on July 14, 2010, 09:52:55 am
Whilst I agree with some of the points Frank makes, (they are essentially what we recommend in the "VC Best Practices" white paper), I must disagree with Frank's insistence that you always choose "Force Reload from XMI".

Forcing a reload from XMI causes packages and elements to be unnecessarily deleted from the model before re-importing them.

...and deleting the elements from the model, is precisely what is causing the problems with the cross-package dependencies that you are complaining about!

regards,
Howard.
Title: Re: Bug in GetAllLatest
Post by: dschmid2 on July 14, 2010, 03:26:00 pm
Actually we were having troubles when NOT doing a force Reload from XMI, we used to have missing dependencies and even missing classes. After starting to use the forced reload (twice in a row), things went somehow better. But as a matter of facts, with both methods we end up from time to time with models that were corrupt.

Since we encountered this issue with both methods, there this fear of never knowing when its going to happen next.

Worst of all, there is no evidence or message alerting that something could be missing, not logfile, just nothing. It can happen that only some time later (and after lots of modification and check-ins/outs) we get aware that something must have been lost on the way. And from that point on, its a real pain to reconstruct things...

Title: Re: Bug in GetAllLatest
Post by: Frank Horn on July 14, 2010, 05:37:50 pm
Howard,

I agree that forcing reload from xmi is unnecessary when all users know what they're doing and everyone is duly updating packages in the right order.

But the tiniest flaw in this workflow can lead to loss of data.

Imagine we are both working on a project with two controlled packages A and B, where B depends on A, and we both have a local eap file containing A and B.

Now you check out both A and B, add some classes to A, and associations from B-classes to A-classes. Then you check in both packages.

Then I open my project to work on package B. Here's what can happen:

Case 1: Everything allright
I've received your notification and I know that B depends on A. So I call "Get Latest" first on A and then on B. EA will know that the packages are out of date and reload from xmi anyway, so I don't have to force it. Everything will be fine.

Case 2: Just made it
I know that you have worked on B, but I'm not aware of the changes you made to A, or I don't know that I have to take care of this. I check out B and don't see your changes. Now I wonder what's going on and call "Get Latest" for A as well, just to be on the safe side. Still no changes in B. Then I call "Get Latest" for B again, but EA tells me it's up to date. Only "Undo Checkout" or "Get Latest" for B with forced reload will get your changes into my model.

By the way, what if I call "Get All Latest" on the whole project? Can I trust EA to import first A and then B? Or do I have to call "Get All Latest" twice, the second time forcing reload?

Case 3: Messed up
I don't know anything, check out B, and spend the day working on B. Then I check in B. Next time you checkt out B, your changes will have gone down the drain.

I think EA should at least give a warning when things are being deleted on import. Better still, it should keep stubs for all such things, in all cases.
Title: Re: Bug in GetAllLatest
Post by: dschmid2 on July 15, 2010, 03:33:28 pm
Dear guys from EA

During the GetAllLatest operation, I noticed several times, that in the progress window, it shows a line telling "Error ...." but I never managed to read the full content, because the operation went on. So I know that somewhere something is wrong, but I have no clue what EA is complaining about...

PLEASE! Add a simple logging functionality, that prints in the System Message window all the packages that have been imported during a GetAllLatest operation. Please dump the filename, package name the repository version and eventual problems. This can't be such a big thing. And it would help us to at least verify which packages were updated, determine which classes were replaced etc.

Thanks a lot!!! ;D
Title: Re: Bug in GetAllLatest
Post by: Geert Bellekens on July 15, 2010, 04:03:13 pm
Have you oficially requested this feature?
Although posting on this forum is very interesting, it won't put the feature on Sparx TODO list.
Only an official feature request using the link (http://www.sparxsystems.com/support/feature_request.html) on the bottom of the page will.

Geert
Title: Re: Bug in GetAllLatest
Post by: suki on April 18, 2011, 11:01:16 pm
Hello,
we use EA 864 and in this version is still the problem decribed above.

We work with local EAPs (many users) a have one central storage in subversion.
For us  is not possible to keep those difficult rules described by Frank because we are more than 50 users who works with it.

Has anyone functioning solution how to update local EAPs from XMI files from Subversion without loosing relation between objects and damage of diagrams?

Some bugs can be repared running GetAllLatest twice but not all of them except the fact that run of GetAllLatest on our great database takes more than one hour. It is unusable :-(.

Thanks in advance for all comments.
Title: Re: Bug in GetAllLatest
Post by: HowardB on April 19, 2011, 09:46:37 am
Hello Suki,

New functionality has been introduced into the Get All Latest command in EA version 9.0, that addresses the issues of elements and relationships being removed from diagrams during XMI imports.

EA 9.0 (beta 2) is available now, for download by registered users.

If your project is so large that it takes around an hour to run Get All Latest, maybe you should consider hosting the project in a DBMS, such as MySQL or SQL Server.  In that case, you would have no need to run Get All Latest, as all users would be connecting to the same model database, and always accessing the latest model information.

best regards,
Howard Britten.
Title: Re: Bug in GetAllLatest
Post by: KP on April 19, 2011, 10:28:59 am
Quote
EA 9.0 (beta 2) is available now, for download by registered users.
Beta 2 is actually a public beta - there is a 30 day time-limited trial version for unregistered users. Follow the links on the front page...
Title: Re: Bug in GetAllLatest
Post by: lubos on April 19, 2011, 05:47:14 pm
Hello,

could you describe what are the changes in version 9 in more detail, please? This info is necessary for us to decide if the new version is suitable in our environment and way of work.
I try to avoid situation when we will trust this new version and a new problems with data/work lost will arrise...

Thank you
 
Title: Re: Bug in GetAllLatest
Post by: suki on April 19, 2011, 06:28:21 pm
Thank you for your comment.
We know that Beta 9.0 is available but they would be useful more details in change log (see lubos comment).
From my point of view is better to wait for official release than work with beta on production environment, do you agree?

Concerning your DBMS recomendation. We worked with DBMS in past so I know advantages and disadvatages DBMS.

Our users work on different places often using internet connection only. So work with database was very slow therefore we moved to local EAPs with VC.
Work with EAPs is much more faster for routine work and it allows to manage exclusive access to the package. This is what we exactly need. Unfortunatelly synchronization of versioned packages is nearly unusable. With each update we are losing some work documented in past.

suki
Title: Re: Bug in GetAllLatest
Post by: Eve on April 20, 2011, 08:30:35 am
Quote
From my point of view is better to wait for official release than work with beta on production environment, do you agree?
Yes, I agree. Sparx Systems does not endorse using a beta in production environments.

If you were going to test this I would recommend setting up an additional model to connect to the same version control provider so that you could verify that the change does help in this situation.
Title: Re: Bug in GetAllLatest
Post by: HowardB on April 20, 2011, 10:42:33 am
As I mentioned in an earlier post, the Get All Latest command suffered from an issue where information that was deleted from the model when re-importing a package, could not be properly restored to diagrams contained in other packages.  (The constructs were still recorded in the model, but unfortunately, the depiction of those contructs was not being restored to all of the diagrams.)
Diagrams contained in the re-imported package were fully restored, but diagrams in other packages could be affected.  The order of re-importing the packages determined which diagrams were affected.

New functionality has been added to the Get All Latest command, such that AFTER all of the package imports have been completed, EA scans all of the XMI files associated with the model's packages, to ensure that everything listed in the XMI, has in fact been restored to the model.  Any information that is found to be missing from diagrams is restored at this point.  This extra process is carried out AFTER the normal round of deleting and re-importing has been completed.

The end result is that the information in the model database, is a true representation of the information recorded in the version controlled package XMI files.

I hope that helps.

best regards,
Howard Britten.
Title: Re: Bug in GetAllLatest
Post by: lubos on April 20, 2011, 03:29:02 pm
I believe you should do more radical changes to the import process.
I liked the concept of Rational Rose (the case tool was terrible but the importing process was good) --- you shouldn't delete any references to not known elements in the model because you are not sure if the model is complete. This can solve the problem of big projects when I can import only a required part of the model  to EAP to work on it, but I can be sure I will not lost any references to not loaded parts of the model.
You can provide some "clean up" functionality to delete all external references to unknown elements in the models marked as "fully loaded".

Do you think these changes can be useful too?

Lubos