Thank you Konstantin for your detailed feedback. I have included the developer, Sarika Sinha, in this email chain.
Please see my responses in green
I love seeing improvement in this area and workflow like the one described would be a big usability improvement. Having said that, I am concerned that there isn’t sufficient time left in Mars.1 development cycle for adopters to fully test, provide feedback and get bug fixes. I’d rather see this iterated on in the Neon release.
The idea behind the fix is to ensure existing server adapter providers are not broken and that they do not need to react to this new change. If the providers choose to react, then their changes will get picked up. If not, then nothing will be broken (e.g. their server adapter will still show up and install as before).
[kosta] I understand that this is the intention, but there are a lot of details and corner cases to get right here. If UX isn’t quite right or there are caching/network glitches, then adopters are going to be dealing with user support cases when users aren’t able to installer the adapters. What is the driver for rushing this out in Mars.1 without taking the time to ensure that the feature is robust and polished?
[sghung] Mars SR1 is still a release that allows for enhancements. By providing a method of reverting to the old behaviour, we ensure the corner cases and glitches can be resolved. In addition, as users are exposed to this enhancement, the feedback can be used towards improving this enhancement in SR2 and beyond.
My feedback and questions:
1. The wiki only mentions the New Server wizard. Is it safe to assume that similar changes would be made to the New Runtime wizard?
No, this feature will not be in the Runtime wizard. Having it in the New Server Wizard provides a better flow for the user, as it is easier to access. To access the Runtime configuration page (as there is no Runtime wizard), the user has to open the Preferences page. This preference page is less visible than creating a new server as the Servers view is opened by default in the Java EE perspective, so the user can more easily create a server than a runtime.
[kosta] I see this disparity causing a lot of user confusion as this page of the New Runtime and New Server wizards has before looked and behaved the same. This feature should be extended to the New Runtime wizard.
[sghung] Having the two wizards be the same sounds like a much larger enhancement, which is outside of the scope of this particular enhancement. Even before this enhancement, I would argue that the two wizards are not the same. The New Server Wizard would link back to the Runtime preferences page for server adapter that already exists. If it does not exist, there is no linkage.
2. Install feasibility validation should not be performed on selection of item in the tree. No expensive operation should be performed on this event as users have a habit of clicking around while exploring. A better approach would be to always enable the Next button and only validate when user elects to move forward. Wizard progress bar should be used while validation is in progress. If successful, the user gets the license screen. If not successful, the user gets a page with failure details. The current approach of directing user to the Error Log is not a good practice. The Error Log should not be used for communicating expected failures.
Thanks for your suggestion, Sarika will try and implement this tomorrow.
[sghung] This issue has been fixed. Clicking on the tree item no longer performs the validation. Once the user clicks on the desired downloadable server adapter, the user would click "Next". Only then, will the server adapter perform the validation. If there is an error, the error message (which includes the diagnostic information) will be printed out to the user on the wizard page.
3. When is the list of downloadable adapters initially fetched and is there visible feedback that fetch is in progress?
The list of downloadable adapters initially fetched is a snapshot created at the time of this feature's development (mid August 2015). This cache is used to ensure the New Server wizard loads quickly.
[kosta] Having a pre-fetched cache in WTP distribution does avoid the problem of the initial fetch, but what is the process for updating the static snapshot? Is someone going to have to remember to do this for every WTP release?
[sghung] For Mars SR1, this method of using a snapshot is the current approach. For Mars SR2, I have confirmed with the developers that they will work on an approach that will do an automatic cache update to avoid this stale cache problem. Currently, if the user does click on a downloadable server adapter, that particular entry will be refreshed automatically (without requiring the user to click the "Refresh" button). That is, if the server's update site has an updated version, this new version would be installed (not the timestamp and version that is stored in the cache).
When the "Refresh" button is pressed, there is progress in the Progress view showing the fetching. The progress is not in the wizard, but the user will see a background process showing up in the progress bar on the lower right corner.
[kosta] If user initiates an action, especially from within a dialog, the progress feedback needs to be more obvious than the jobs progress indicator. A typical UX pattern employed elsewhere in Eclipse is to run the task in the progress dialog that has a “run in the background” button.
[sghung] I agree that showing progress would be better. Currently, the "Refresh" button becomes disabled while the refresh is occurring. The behaviour is similar to when the user modifies the compiler settings (e.g. validation). The job will be kicked off in the background and if the user closes the preference page, the progress dialog will appear with the "Run in background" button. I will bring up this concern to the developer. I feel like having the progress bar in the wizard (instead of a separate dialog) would be more ideal.
4. Is there feedback when downloadable adapters list could not be downloaded? We need to ensure that we avoid user confusion when they don’t see their server listed due to network issues.
On failure, there is a message in the Logs view to help the user diagnose the issue.
[kosta] Using the Error Log view for expected failures messages is a bad UX practice. In the case of a failure of a manually-activated action, any failure should be displayed in a message box.
[sghung] This issue has been fixed and the explanation of how it works is seen on point #2.
5. What exactly is cached? These details are needed to help evaluate the likelihood that the cached data will be stale when a new version of an existing adapter is released.
The cache contains the following information: feature description, feature id, feature name, feature provider, and feature version. For example:
<feature description="Tools for developing applications for Oracle WebLogic Server (versions 12.1.x, 10.3.x, 10.0 and 9.2)." id="oracle.eclipse.tools.weblogic.feature.group" name="Oracle WebLogic Server Tools" provider="Oracle" uri="http://download.oracle.com/otn_software/oepe/mars/" version="126.96.36.199507100122"/>
[kosta] What is the purpose of caching the version? This bit of data will grow stale the quickest. What happens when the cache contains an older version than what’s currently published?
[sghung] The version is cached in order to allow for future validation and discovery of downloadable adapters. By having the framework in place with this extra information, it will also help with the automatic refresh of the cache in the future. Currently, the version in the cache does not have any effect.
6. Is there a policy for updating cached data without having the user click on the refresh button? I would recommend triggering a refresh job when server tools is initially activated as well as after certain number of days have expired in case the user keeps Eclipse open for a long time.
No, currently there is no policy for updating the cache without having the user click the button. Kicking off the job in the background can take more than one minute, so it is more ideal to have an initial cache (a snapshot from mid August 2015) and provide a way for the user to update the cache themselves (by clicking the "Refresh" button).
[kosta] Automated cache refresh can happen in the background on Eclipse startup and provides a better user experience than requiring the user to hit the refresh button manually, especially once the distribution is a bit older and the cache is so out of date that every new installation _requires_ the user to hit refresh before they can use the feature.
[sghung] The automatic caching mechanism will be examined as part of Mars SR2 by the developers. The fix will remove the need for the "Refresh" button and out-of-date cache.
7. The installation seems to use a separate progress dialog box. It would be best to use the built-in progress bar of the wizard.
The separate progress dialog existed before. To add this into the wizard would be something to investigate after Mars SR1.
8. There is a terminology mismatch caused by mixing feature names and server names in the same tree. Whereas all entries should be a server type, some are “X Server Adapter” or “X Tools”. This could be solved by adding metadata to the adapter repo so that it advertises the list of supported server types.
The terminology mismatch is caused by the server adapter's metadata. It will be up to the server adapter providers to provide the correct metadata (but we are not forcing them to do so right now).
[kosta] You are using feature name in place of server name. The two will never match. It’s not something that adapter providers can fix on their own. For instance, the dialog needs “WebLogic Server”, but the feature name will always be “WebLogic Server Tools” or “WebLogic Server Adapter”. Additional metadata properties need to be defined to allow adapter providers to supply you with server names as opposed to feature names.
[sghung] For comments #8-10, thank you for elaborating on this metadata issue. I will talk with the developers on this. In the future, the metadata to provide the correct labels and what server would be installed would be there. However, even in the future, the need for showing the downloadable server if the server adapter does not react still needs to be in place. By using the currently existing metadata, it can populate the New Server wizard with its best effort.
9. Many of the downloadable adapters support multiple server versions. This leads to the contents of the tree changing after the install is completed. Ideally, only downloadable indicator icon would change. This can be solved with additional metadata like #8.
Yes, the metadata provider can change how the server appears in this list.
[kosta] I don’t understand this answer. I am looking for a solution for the case where one adapter feature handles multiple server versions and the provider wants to keep it as one feature.
10. How is vendor/category determined for downloadable adapters?
The vendor/category is taken from the metadata retrieved from querying for the server adapter. For point #5, I have pasted an example of a query where we get the feature provider (in that case, it is Oracle). That is used as the vendor/category.
[kosta] That sort of works. You are conflating adapter vendor with server vendor. The two don’t have to be the same, even if they are the same for all current examples.
11. Ideally, the user should be able to resume where they left off in the wizard after the download completes and Eclipse restarts. Upon the restart, the wizard should restart with the desired server type pre-selected and page advanced to the next page, as if the download didn’t need to happen. This requires #9 in order to know which server type/version the user intended to create.
This issue is more of a limitation on the Eclipse platform. It's definitely a nice to have and would bring convenience to this item. However, this feature would be outside of the scope of this fix for SR1.
[kosta] Without ability to resume, the UX strikes me as unfinished. It contains the promise that the user can install as part of the wizard flow without actually allowing the flow to resume properly after restart. This UX issue didn’t exist in the old UI as the action to install the adapter was a separate command outside the main flow of the wizard.
[sghung] Getting back to the New Server wizard after a refresh is tricky. As mentioned before, this seems to be a limitation on the Eclipse platform. For instance, I know of an extender of WTP, which will bring up a dialog on startup (in some situations) or a different welcome page. As a result, the New Server wizard automatically coming up again would create multiple wizards shown at the same time. I believe something needs to be changed on the Eclipse platform side to create a proper snapshot of what's happening and then bring up the exact same snapshot on a restart.
12. What happens to the “New Server”/”New Runtime” wizard if the user chooses not to restart?
A restart is required whenever the user installs a new server adapter (e.g. Eclipse marketplace or through the Install New Software). Otherwise, the server will not show up in the New Server Wizard.
The new server type feature is marked as installed in the current configuration only after a restart. As a result, the server will show up in the wizard as "downloadable" and allow users to install (again). The behavior is similar to the old behavior if you don't restart - the newly installed Server Type will not be available until restart. If the user selects and installs it again, it successfully completes and ask for restart (this time around it goes faster as the artifacts are present locally).
[kosta] My question pertains specifically to the wizard? Does the wizard exit?
[sghung] yes, the wizard exits when the installation has completed and requires a restart.
13. Why is there a hyperlink on “Show additional server adapters”?
This hyperlink exists to provide the old behaviour. In case the user wants to go through the old flow, the user can uncheck the checkbok. The hyperlink will be enabled and click on the hyperlink will launch the old wizard
[kosta] That’s pretty confusing UX. Why is it necessary to preserve the old UX? In my mind, if the new UX is working properly there is no need for the old one. Multiple UX paths will only serve to confuse users.
[sghung] At least for the initial release in SR1, preserving the old behaviour is desired in case the user does not like the new approach. By getting feedback from the initial release in SR1, these issues can be addressed in SR2 and beyond.
14. The “Show additional server adapters” checkbox and the refresh button take up valuable prime real estate in the wizard, yet most users should not have a cause to interact with these controls. I suggest moving these to preferences.
The real estate of this dialog has not changed significantly with this change. The old dialog had a hyperlink at the top of the New Server wizard too:
[kosta] In the old UX, the hyperlink was essential to access the install facility. Once UX and caching issues in the new approach are smoothed out, users aren’t going to need to interact with those controls except in very rare cases. As such, there is little need to continue to take up the space in the wizard.
[sghung] This issue will be fixed as described in point #6.
Going to the Preferences page is a lot harder to find for the user and the "Refresh" button approach is still preferred, so it makes more sense to keep these features in the New Server Wizard.
[kosta] The user should not have to hit Refresh button at all, except in some very rare corner cases, so there is no need to have it in a prominent location.
[sghung] This issue will be fixed as described in point #6.
Thanks for your consideration of this feedback.