How we made an application for Windows 10 with Fluent Design (UWP / C #)

    We at ivi have long been planning to update our application for Windows 10 (which is for PCs and tablets). We wanted to make it a kind of "cozy" corner for relaxation. And therefore, the recently announced Microsoft concept of fluent design  came in handy for us.

    But I will not talk about standard components here, we offer Microsoft for fluent design ( Acrylic , Reveal , Connected animations  , etc.), although we, of course, use them too. With them, everything is simple and clear - take the documentation and use it. 

    But adventures usually begin when you leave the beaten track. Therefore, I’d better tell you about how we did one custom control, which gave us a lot of trouble. Here is one:

    image

    The idea is that we use depth and motion from the fluent design system. The central element, as it were, slightly rises above all the others. This is achieved by animating its size and shadow while scrolling. 

    The FlipView control  didn’t come right away, because he cannot show pieces of the next and previous elements (we call them “ears”). And we began the search for a solution.

    Way 1. We try to use GridView


    The logical solution was to try using GridView . To line items in a horizontal line, set as ItemsPanel set:


    To center the current element, use the ScrollViewer properties in the GridView template: 


    An example of such an implementation can be seen, for example, here .

    Everything seems to be ok, but there are problems.

    Gridview Problem 1. scaling control over the entire width of the screen


    According to the idea of ​​designers, our control should stretch to the full width of the window. This in itself is not a problem. But when changing the size of the control, the sizes of all its children (Items) must also be synchronously changed:

    • The width of the elements must be set to 90% of the control width (10% left on the “ears”);
    • The height of the elements must be calculated based on the width and proportions of the image;
    • At the same time, on the small screen width, you need to crop the image on the left and right so that it does not become too small after scaling.

    image

    Out of the box, GridView doesn't. We spotted the solution in the implementation of the AdaptiveGridView control  from UWPToolkit :

    • We will inherit from GridView and we add two properties: ItemWidth and ItemHeight;
    • In the SizeChanged event handler, we calculate these properties depending on the width of the GridView;
    • Overriding the PrepareContainerForItemOverride method on the GridView. It is called for each ItemContainer before it is shown to the user. And we add for each item the bindings to the ItemWidth and ItemHeight we created:

    protected override void PrepareContainerForItemOverride(DependencyObject obj, object item)
    {
        base.PrepareContainerForItemOverride(obj, item);
        if (obj is FrameworkElement element)
        {
            var heightBinding = new Binding()
            {
                Source = this,
                Path = new PropertyPath("ItemHeight"),
                Mode = BindingMode.TwoWay
            };
            var widthBinding = new Binding()
            {
                Source = this,
                Path = new PropertyPath("ItemWidth"),
                Mode = BindingMode.TwoWay
            };
            element.SetBinding(HeightProperty, heightBinding);
            element.SetBinding(WidthProperty, widthBinding);
        }
    }

    For more details, see the UWPToolkit source code .

    It seems everything is OK, it works. But…

    Gridview Problem 2. When resizing items, the current item goes out of scope


    But as soon as we begin to dynamically change the width of the elements inside the GridView, we are faced with the following problem. At this point, a completely different element begins to fall into the visible region. This is due to the fact that the HorizontalOffset of the ScrollViewer inside the GridView remains unchanged. GridView does not suggest such a catch from us. 
    image

    The effect is especially noticeable when maximizing the window (due to a sharp change in size). And even with just large values ​​of HorizontalOffset.

    It would seem that this problem could be solved by asking the GridView to scroll to the desired element:

    private async void OnSizeChanged(object sender, SizeChangedEventArgs e)
    {
        ...
        await Task.Yield();
        this.ScrollIntoView(getCurrentItem(), ScrollIntoViewAlignment.Default);
    }

    But no:

    • Without using Task.Yield (), this does not work. And with it - leads to an ugly visual twitch - because another element manages to be displayed before ScrollIntoView is executed.
    • And when SnapPoints is enabled, ScrollIntoView for some reason, in principle, does not work correctly. As if stuck on them.

    This could also be solved by manually calculating and setting a new HorizontalOffset value for ScrollViewer each time we resize our GridView:

    private void OnSizeChanged(object sender, SizeChangedEventArgs e)
    {
        ...
        var scrollViewer = this.FindDescendant();
        scrollViewer.ChangeView(calculateNewHorizontalOffset(...), 0, 1, true);
    }

    But this only works with a gradual resizing of the window. When maximizing the window, this often gives the wrong result. Most likely, the reason is that the new HorizontalOffset value that we calculated turns out to be too large and goes beyond the ExtentWidth (the width of the content inside the ScrollViewer). And since GridView uses UI-virtualization , then automatically after changing the width of the ExtentWidth Item-s may not be recounted. 
     
    In general, an adequate solution to this problem could not be found. 

    We could stop here and start searching for the next solution. But I will describe another problem with this approach.

    Gridview Problem 3. Nested ScrollViewers break horizontal scrolling with the mouse


    We want the mouse wheel to always scroll vertically. Is always. Vertically.

    But if we put on the GridView page with horizontal scrolling, then the ScrollViewer located in its depths captures the mouse wheel events and does not skip higher. As a result, if the mouse cursor is located above our control-list, the mouse wheel makes a horizontal scroll in it. This is inconvenient and confusing for users: There

    image

    are two solutions to this problem:

    • Catch the PointerWheelChanged event before it hits the horizontal ScrollViewer and, in response to it, call ChangeView () on the vertical ScrollViewer. Spied here . This works, but noticeably slows down when the mouse wheel rotates quickly. It didn’t suit us - spoiling the scroll with the mouse for rare users with a touch screen is not an option.
    • Install  HorizontalScrollMode="Disabled". This helps, but it disables not only the mouse wheel, but scrolling through the touch screen.

       

    We didn’t want to lose the Touch screen, and we continued to search for a better solution.

    Path 2. Carousel from UWPToolkit 


    The next solution was the Carousel control  from UWPToolkit . From all sides a very interesting and informative control. I recommend everyone to study its implementation.

    He covered our needs pretty well. But in the end, too, did not fit:

    • There is no scaling of elements when changing the width (see above):

      • This is a solvable problem. Because he is open source. And adding scaling to it will not be difficult. 
      • And even the problem of keeping the current element in scope after scaling is also solved, again thanks to the open source implementation.
    • Missing UI virtualization :

      • Carousel uses its own implementation of ItemsPanel. And there is no support for UI virtualization in it;
      • This is quite a critical thing for us, because we can have quite a lot of promotional materials in the leaflet and this greatly affects the page load time;
      • Yes, this is probably also feasible. But it no longer looks simple.
    • It uses animations on a UI thread (Storyboards and Manipulation * events), which, by definition, is not always smooth enough.

    Those. it turns out that you need to spend quite a lot of time finalizing this control to our needs (which is worth one UI virtualization). At the same time, at the output we get a piece with potentially slowing down animations. 

    In general, we also decided to abandon this approach. If we waste time, then we will do it wisely.

    Way 3. Our implementation


    Making “TOUCH ONLY” ScrollViewer


    Let me remind you that we do not want to use the standard ScrollViewer, due to the fact that it captures all events from the mouse wheel (see the section "GridView. Problem 3" above).

    We don't like the implementation from Carousel, as uses animations on a UI stream, and the preferred method for creating animations for UWP applications is  Composition animations . Their difference from the more familiar Storyboards is that they work on a separate Composition stream and, due to this, provide 60 frames / sec even when the UI stream is busy with something.

    To accomplish our task, we need  InteractionTracker - a component that allows you to use touch-input as a source for animations. Actually, the first thing we need to learn to do is move the UI elements horizontally depending on the movement of the finger across the screen. In fact, we will have to start by implementing our custom ScrollViewer. So let's call it TouchOnlyScrollViewer:

    public class TouchOnlyScrollerViewer : ContentControl
    {
        private Visual _thisVisual;
        private Compositor _compositor;
        private InteractionTracker _tracker;
        private VisualInteractionSource _interactionSource;
        private ExpressionAnimation _positionExpression;
        private InteractionTrackerOwner _interactionTrackerOwner;
        public double HorizontalOffset { get; private set; }
        public event Action ViewChanging;
        public event Action ViewChanged;
        public TouchOnlyScrollerViewer()
        {
            initInteractionTracker();
            Loaded += onLoaded;
            PointerPressed += onPointerPressed;
        }
        private void initInteractionTracker()
        {
            // Инициализируем InteractionTracker и VisualInteractionSource
            _thisVisual = ElementCompositionPreview.GetElementVisual(this);
            _compositor = _thisVisual.Compositor;
            _tracker = InteractionTracker.Create(_compositor);
            _interactionSource = VisualInteractionSource.Create(_thisVisual);
            _interactionSource.PositionXSourceMode =
               InteractionSourceMode.EnabledWithInertia;
            _tracker.InteractionSources.Add(_interactionSource);
            // Создаём тривиальную Expression-анимацию, которая в качестве источника 
            // использует touch-смещение из InteractionTracker
            _positionExpression = 
               _compositor.CreateExpressionAnimation("-tracker.Position");
            _positionExpression.SetReferenceParameter("tracker", _tracker);
        }
        private void onLoaded(object sender, RoutedEventArgs e)
        {
            // Привязываем нашу анимацию к свойству Offset дочернего UIElement-а
            var visual = ElementCompositionPreview.GetElementVisual((UIElement)Content);
            visual.StartAnimation("Offset", _positionExpression);
        }
        private void onPointerPressed(object sender, PointerRoutedEventArgs e)
        {
            // перенаправляем touch-ввод в composition-поток
            if (e.Pointer.PointerDeviceType == PointerDeviceType.Touch)
            {
                try
                {
                    _interactionSource.TryRedirectForManipulation(e.GetCurrentPoint(this));
                }
                catch (Exception ex)
                {
                    Debug.WriteLine("TryRedirectForManipulation: " + ex.ToString());
                }
            }
        }
    }

    Here, everything is strictly according to the dock from Mircosoft. Unless the TryRedirectForManipulation call had to be wrapped in try-catch because it sometimes throws sudden exceptions. This happens quite rarely (offhand, in about 2-5% of cases) and we were not able to find out the reason. Why nothing is said about this in the documentation and official examples of Microsoft - we do not know;)

    TOUCH ONLY ScrollViewer. We form HorizontalOffset and events ViewChanging and ViewChanged


    Since we are making a resemblance to a ScrollViewer, we need the HorizontalOffset property and the ViewChanging and ViewChanged events. We will implement them through processing callbacks of InteractionTracker. To obtain them, when creating an InteractionTracker, you must specify an object that implements IInteractionTrackerOwner, which will receive these callbacks:

    _interactionTrackerOwner = new InteractionTrackerOwner(this);
    _tracker = InteractionTracker.CreateWithOwner(_compositor, _interactionTrackerOwner);

    For completeness, let me copy a picture from the documentation with the states and events of InteractionTracker:
    image

    The ViewChanged event will be thrown upon entering the Idle state.

    The ViewChanging event will be thrown when IInteractionTrackerOwner.ValuesChanged is fired.
    I must say right away that ValuesChanged can happen when the InteractionTracker is in the Idle state. This happens after calling InteractionTracker TryUpdatePosition. And it looks like a bug in the UWP platform.

    Well, you have to put up with it. Fortunately, it’s not difficult for us - in response to ValuesChanged we will throw out either ViewChanging or ValuesChanged, depending on the current state:

    private class InteractionTrackerOwner : IInteractionTrackerOwner
    {
        private readonly TouchOnlyScrollerViewer _scrollViewer;
        public void ValuesChanged(InteractionTracker sender,
                                  InteractionTrackerValuesChangedArgs args)
        {
            // Сохраняем текущее смещение. Пригодится для будущих нужд.
            _scrollViewer.HorizontalOffset = args.Position.X;
            if (_interactionTrackerState != InteractionTrackerState.Idle)
            {
                _scrollViewer.ViewChanging?.Invoke(args.Position.X);
            }
            else
            {
                _scrollViewer.ViewChanged?.Invoke(args.Position.X);
            }
        }
        public void IdleStateEntered(InteractionTracker sender,
                                     InteractionTrackerIdleStateEnteredArgs args)
        {
            // Здесь нельзя использовать _scrollViewer._tracker.Position. 
            // В Windows 14393 (Anniversary Update) он почему-то всегда 0
            _scrollViewer.ViewChanged?.Invoke(_scrollViewer.HorizontalOffset, requestType);
        }
    }

    TOUCH ONLY ScrollViewer. Snap Points to scroll exactly 1 element


    To ensure scrolling exactly on 1 element, there is a wonderful solution - “ snap points with inertia modifiers ”.

    The point is that we set the points at which scrolling has the right to stop after swiping on the touch screen. And the rest of the logic is taken by InteractionTracker. In fact, it modifies the deceleration rate so that the stop after the swipe occurs smoothly and at the same time exactly where we need it.

    Our implementation is slightly different from that described in the example in the documentation . Because we don’t want to scroll more than one element at a time, even if the user “twisted” our leaflet too quickly. 

    Therefore, we add only three snap-points - “one step to the left”, “one step to the right” and “stay in the current position”. And after each scrolling we will update them.

    And in order not to recreate snap points every time after scrolling, we will make them parameterizable. To do this, start a PropertySet with three properties:

        _snapPointProps = _compositor.CreatePropertySet();
        _snapPointProps.InsertScalar("offsetLeft", 0);
        _snapPointProps.InsertScalar("offsetCurrent", 0);
        _snapPointProps.InsertScalar("offsetRight", 0);

    And in the formulas for Condition and RestingValue we use the properties from this PropertySet:

        // Точка привязки на «на один шаг влево»
        var leftSnap = InteractionTrackerInertiaRestingValue.Create(_compositor);
        leftSnap.Condition = _compositor.CreateExpressionAnimation(
           "this.Target.NaturalRestingPosition.x < " +    
           "props.offsetLeft * 0.25 + props.offsetCurrent * 0.75");
        leftSnap.Condition.SetReferenceParameter("props", _snapPointProps);
        leftSnap.RestingValue = 
           _compositor.CreateExpressionAnimation("props.offsetLeft");
        leftSnap.RestingValue.SetReferenceParameter("props", _snapPointProps);
        // Точка привязки на «на один шаг вправо»
        var currentSnap = InteractionTrackerInertiaRestingValue.Create(_compositor);
        currentSnap.Condition = _compositor.CreateExpressionAnimation(
            "this.Target.NaturalRestingPosition.x >= " +
                "props.offsetLeft * 0.25 + props.offsetCurrent * 0.75 && " +
            "this.Target.NaturalRestingPosition.x < " +
                "props.offsetCurrent * 0.75 + props.offsetRight * 0.25");
        currentSnap.Condition.SetReferenceParameter("props", _snapPointProps);
        currentSnap.RestingValue = 
            _compositor.CreateExpressionAnimation("props.offsetCurrent");
        currentSnap.RestingValue.SetReferenceParameter("props", _snapPointProps);
        // Точка привязки «на один шаг вправо»
        var rightSnap = InteractionTrackerInertiaRestingValue.Create(_compositor);
        rightSnap.Condition = _compositor.CreateExpressionAnimation(
            "this.Target.NaturalRestingPosition.x >= " +
            "props.offsetCurrent * 0.75 + props.offsetRight * 0.25");
        rightSnap.Condition.SetReferenceParameter("props", _snapPointProps);
        rightSnap.RestingValue = 
             _compositor.CreateExpressionAnimation("props.offsetRight");
        rightSnap.RestingValue.SetReferenceParameter("props", _snapPointProps);
        _tracker.ConfigurePositionXInertiaModifiers(
            new InteractionTrackerInertiaModifier[] { leftSnap, currentSnap, rightSnap });
    }

    Here:

    • NaturalRestingPosition.X is the offset at which the inertia would end if there were no snap points;
    • SnapPoint.RestingValue - the offset at which stopping is allowed when the SnapPoint.Condition condition is met.

    First, we tried to set the border in Condition in the middle between the snap points, but users noticed that for some reason not every swipe caused scrolling to the next element. Some swipes were not fast enough and there was a rollback.

    Therefore, in the formulas for Contition, we use the coefficients 0.25 and 0.75, so that even a “slow” swipe scrolls to the next element. 

    Well, after each scrolling to the next element, we will call this method to update the snap point parameters:

    public void SetSnapPoints(double left, double current, double right)
    {
        _snapPointProps.InsertScalar("offsetLeft", (float)Math.Max(left, 0));
        _snapPointProps.InsertScalar("offsetCurrent", (float)current);
        _snapPointProps.InsertScalar("offsetRight",
            (float)Math.Min(right, _tracker.MaxPosition.X));
    }

    UI virtualization dashboard


    The next step was to build a full-fledged ItemsControl based on our TouchOnlyScrollerViewer.

    For reference. UI virtualization is when a control list, instead of, say, 1000 children, creates only those that are visible on the screen. And reuses them as you scroll, binding to new data objects. This allows you to reduce page load time with a large number of items in the list.

    Because after all, I really didn’t want to implement our UI virtualization, the first thing we, of course, tried to do was to use the standard ItemsStackPanel .

    I wanted to make her friends with our TouchOnlyScrollerViewer. Unfortunately, it was not possible to find any documentation about its internal device or source code. But a series of experiments suggested that the ItemsStackPanel looks for the ScrollViewer in the Visual Tree in the list of parent elements. And somehow override this, so that instead of the standard ScrollViewer it would be ours, we would not find.

    Well. So the panel with UI virtualization will still have to be done independently. The best that was found on this topic was this series of articles as early as 11 years ago:  one , two , three , four . True, it is about WPF, and not about UWP, but it conveys the idea very well. We took advantage of it.

    Actually, the idea is simple:

    • Such a panel is embedded inside our TouchOnlyScrollerViewer and subscribes to its ViewChanging and ViewChanged events;
    • The panel creates a limited number of child UI elements. In our case, this is 5 (one in the center, two on the “ears” protruding from the left and right, and another 2 for the cache of the elements following the “ears”);
    • UI elements are positioned depending on TouchOnlyScrollerViewer.HorizontalOffset and are re-attached to the desired data objects as they are scrolled.

    I will not show the implementation, because it turned out quite complicated. It is rather a topic for a separate article.

    We are looking for Tapped events lost after redirecting touch input to the composition stream


    After we put it together, another interesting problem was revealed. Sometimes the user taps on the elements inside our control while the touch input is redirected to InteractionTracker. This happens when inertia scrolling occurs. In this case, the PointerPressed, PointerReleased, and Tapped events just don't happen. And this is not a far-fetched problem, because InteractionTracker’s inertia is rather long. And even when visually scrolling is almost over, in fact, slow scrolling of the last few pixels may occur.

    As a result, the user is upset - he expects that the page of the selected movie will open on tap. But this does not happen.

    Therefore, we will identify tap by a pair of events from InteractionTracker: 

    • Transition to the Interacting state (the finger touched the screen);
    • Then immediately (in less than 150ms) transition to the Inertia state (finger released the screen). And at the same time, the scrolling speed should be zero (otherwise it is no longer a tap, but a swipe):

    public void InertiaStateEntered(InteractionTracker sender,
                                    InteractionTrackerInertiaStateEnteredArgs args)
    {
        if (_interactionTrackerState == InteractionTrackerState.Interacting
            && (DateTime.Now - _lastStateTime) < TimeSpan.FromMilliseconds(150)
            && Math.Abs(args.PositionVelocityInPixelsPerSecond.X) < 1 /* 1px/sec */)
        {
            _scrollViewer.TappedCustom?.Invoke(_scrollViewer.HorizontalOffset);
        }
        _interactionTrackerState = InteractionTrackerState.Inertia;
        _lastStateTime = DateTime.Now;
    }

    It works. But, however, it does not allow to recognize the element by which tap was implemented. In our case, this is not critical, because our elements occupy almost the entire visible width of the TouchOnlyScrollViewer. Therefore, we simply choose the one that is closer to the center. In most cases, this is exactly what you need. So far, no one has even noticed that sometimes tap during scrolling can lead to the wrong place. It's not so easy to catch, even if you know about it;)

    Although in the general case, this is certainly not a complete solution. For a full implementation, I would also have to hit my hit testing. But it is not clear how to do it, because the coordinates of the tap are unknown ...

    Bonus Expression animations for opacity, scale and shadows. To finally become beautiful


    And finally, a cherry on the cake is what it was all about. As we scroll, we want to change the size, shadow, and transparency of the elements. To create the feeling that the one in the center is slightly raised.

    For this we will use Expression-animations . They are also part of the Composition subsystem, operate on a separate thread and therefore do not slow down when the UI thread is busy.

    They are created like this. For the property to be animated, we define an expression that defines the dependence of this property on any other properties. The formula is given as a text string.

    Their charm is that they can be arranged in chains. We will use this:
    image

    The source for all animations will be the offset from the InteractionTracker in pixels. Based on it, for each child UI element we will generate the progress property, which will take values ​​in the range from 0 to 1. And already on the basis of progress, we will calculate all other visual properties.

    So, we form _progressExpression so that it takes on values:

    • 0 - if our element has gone far enough and has reached its minimum size and minimum shadow;
    • 1 - if our element is clearly in a central position, at this moment it has a maximum size, and the shadow shows that it is raised, as it were:

    _progressExpression = _compositor.CreateExpressionAnimation(
       "1 - " +
       "Clamp(Abs(tracker.Position.X - props.offsetWhenSelected), 0, props.maxDistance)"
       + " / props.maxDistance");

    Here:

    • Clamp (val, min, max) is a system function. If val goes beyond min / max, then returns min / max. If it does not exit, returns val.
    • offsetWhenSelected - offset of InteractionTracker, at which the current element is strictly in the center of the visible area;
    • maxDistance - distance at which the element takes a minimum size when removed;
    • tracker is our InteractionTracker.

    Add all these parameters to our Expression animation:

    _progressExpression.SetReferenceParameter("tracker", tracker);
    _props = _compositor.CreatePropertySet();
    _props.InsertScalar("offsetWhenSelected", (float)offsetWhenSelected);
    _props.InsertScalar("maxDistance", getMaxDistanceParam());
    _progressExpression.SetReferenceParameter("props", _props);

    And create a PropertySet with the progress property, which will be evaluated through our _progressExpression. This is necessary in order to build the following animations based on this property:

    _progressProps = _compositor.CreatePropertySet();
    _progressProps.InsertScalar("progress", 0f);
    _progressProps.StartAnimation("progress", _progressExpression);

    Now, based on our progress property, we are already creating real “visual” animations using linear interpolation (Lerp and ColorLerp system functions). A complete list of features that can be used in Expression animations can be found here .

    Scaling:

    _scaleExpression = _compositor.CreateExpressionAnimation(
        "Vector3(Lerp(earUnfocusScale, 1, props.progress), " +
                "Lerp(earUnfocusScale, 1, props.progress), 1)");
    _scaleExpression.SetScalarParameter("earUnfocusScale", (float)_earUnfocusScale);
    _scaleExpression.SetReferenceParameter("props", _progressProps);
    _thisVisual.StartAnimation("Scale", _scaleExpression);

    Shadow Radius:

    _shadowBlurRadiusExpression = _compositor.CreateExpressionAnimation(
        "Lerp(blur1, blur2, props.progress)");
    _shadowBlurRadiusExpression.SetScalarParameter("blur1", ShadowBlurRadius1);
    _shadowBlurRadiusExpression.SetScalarParameter("blur2", ShadowBlurRadius2);
    _shadowBlurRadiusExpression.SetReferenceParameter("props", _progressProps);
    _dropShadow.StartAnimation("BlurRadius", _shadowBlurRadiusExpression);

    Shadow color:

    _shadowColorExpression = _compositor.CreateExpressionAnimation(
        "ColorLerp(color1, color2, props.progress)"))
    _shadowColorExpression.SetColorParameter("color1", ShadowColor1);
    _shadowColorExpression.SetColorParameter("color2", ShadowColor2);
    _shadowColorExpression.SetReferenceParameter("props", _progressProps);
    _dropShadow.StartAnimation("Color", _shadowColorExpression);

    Well, for the remaining properties, the formulas are similar.

    Epilogue


    That's all. In fairness, I must say that this control turned out to be perhaps the most difficult from the point of view of implementation. The rest of the fluent design was much easier :)

    → See how it all works by installing the application .

    Also popular now: