During our just-released checkout usability study, we time and again observed that there’s a direct correlation between how and when users are presented with form validation errors and the pace and effort with which they are able resolve those errors.
While the logic and wording of error messages are very important aspects of the user’s validation error recovery experience, our latest eye-tracking and usability testing shows that in order to perfect the user’s recovery experience, live inline validation must be used.
Definition: “Live inline validation” is where the validity of the user’s inputs are checked live as the user progresses through the form, as opposed to checking the inputs in a lump sum when the user submits the form.
Now, our checkout benchmarks reveal that live inline validation has seen an explosive growth in adoption: in 2012, just 13% of checkouts had live inline validation, whereas in 2016, 60% had implemented it. This makes good sense since the general importance of live inline validation is well documented. In fact, it is surprising that 40% of major e-commerce sites still don’t have inline validation.
Another surprising matter is that the potential implementation pitfalls of inline validation haven’t been highlighted much. During our end-user testing we consistently observe two pitfalls that can end up making live inline validation more frustrating than helpful to the user:
Despite these two implementation details being critical to a positive form validation experience, of the checkouts that have inline validation, 20% fail to address at least one of the above pitfalls.
In this article we’ll cover: 5 core UX benefits observed for live inline validation (particularly relevant to the 40% of sites that still lack it), the 2 pitfalls of live inline validation (especially relevant to the 20% of site implementations that fail to address them), along with a 3rd consideration for those seeking to perfect their inline validation experience.
During our large-scale usability testing we observed 5 distinct benefits to using inline validation:
1) Locating errors is easier. Users are told about the error immediately, meaning that they won’t to the same degree have to come to a full stop on a dedicated error page, to try to figure out where the error is. With live inline validation, users are alerted to errors immediately, and the process of locating the error is thus significantly simplified.
2) Field context is fresh in mind. Because users are alerted to the input being invalid right after typing it, the amount of time needed to correct it decreases significantly, as the input and its context are still fresh in the user’s mind. On a normal after-the-fact error page, the user will often have to recall the field context and re-read their input since it can be several fields and minutes ago the user read the field label and entered the invalid field value.
3) Positive inline validation increases confidence. If positive inline validation is used, then even users without any invalid input will benefit. Positive inline validation removes some cognitive load from users since they don’t have to review and validate the form for errors before submitting it. Instead they can confidently proceed to the next field, resting assured that their input is valid. Positive inline validation can furthermore instill a sense of rapid progression and accomplishment as the user “successfully completes” each field and the site provides them with encouraging visual feedback.
4) Less likely to unintentionally skip required fields. In addition to making sure that inputs are valid, inline validation is also efficient for highlighting omitted required fields as users tab through the form (which we found more than 50% of users generally do during checkout). With inline validation in place, users are notified immediately if users tab through a required field without entering a value – vastly reducing the odds of a user accidentally skipping a required field.
5) Persisting credit card data despite lower-level PCI compliance. One of the ways that e-commerce sites can get away with lower-level PCI compliance is by never letting the user’s credit card data touch their server but instead having the user send their credit card information directly to the payment processor. However, one downside of this is that any validation errors will tend to clear the form input because the e-commerce site has naturally has no way of persisting data they never touch or receive. Inline validation can prevent invalid inputs (e.g. a card number that’s too short, too long, or fails Luhn validation) from being submitted in the first place, and thus avoid the page reload that clears out the user’s entire input.
Despite these rather significant benefits, 40% of sites still lack inline validation. However, inline validation isn’t great by default – there are implementation pitfalls that must be avoided or the inline validation may end up doing more harm than good. And apparently these are pitfalls that even major sites may succumb to – upon benchmarking 50 major e-commerce sites, we found 20% of the sites with inline validation to stumble on at least one of the two following pitfalls.
“Why are you telling me my e-mail address is wrong,” a subject complained, “I haven’t had a chance to fill it all out yet!” – the site’s live inline validation had shown an error message immediately when the subject began typing his e-mail address.
While the point of live inline validation is to alert users of invalid inputs early on, overly aggressive premature validation can result in users being distracted from their typing, feeling unfairly reprimanded, and wasting time looking for errors that aren’t there – hardly a great checkout experience!
Most people don’t like being told they are wrong – especially when they aren’t. Users therefore naturally find it very frustrating and (quite understandably) feel unfairly reprimanded when a site claims they’ve made a mistake before they’ve had a chance to enter a valid input.
Moreover, during testing, the sudden appearance of an error message disrupted many of the subjects in their typing of a perfectly valid input, as they stopped to read and interpret the error. Even worse, some subjects misinterpreted this message to mean that what they had entered so far contained some error and thus began looking for mistakes that weren’t there.
“Why are you telling me my e-mail address is wrong, I haven’t had a chance to fill it all out yet!”
It’s therefore key to fine-tune the implementation of inline validations to avoid them becoming a distraction and annoyance to the vast majority of users. In essence, premature validations must be avoided. More specifically, when users enter a new field – without any existing errors – and begin their typing, the validity of the input should not be checked until the user has had a chance to fully type a valid input. Despite benchmarking the checkout flows of major e-commerce sites, 10% of the sites fail to get this right.
Depending on the type of input, avoiding premature validation means that the validity of each field input should be checked when the user leaves the field or reaches the intended input length.
For most field types, the validity may be checked once the user leaves the field (by binding to the
onblur event of the field). Why not sooner? Because the site has no way to tell if or when the user has finished typing and considers their input “done” or “fully entered”. It’s therefore only safe for the site to conclude that the user considers their input “done” when they leave the field / proceed to the next element in the form.
That said, for certain inputs error messages may be shown while the user is typing. These are inputs with fixed or predictable character lengths and/or formats (such as postal codes, phone numbers, credit card numbers, coupon codes, etc), where the validity can be checked once the user reaches this intended length. This logic can, however, be complicated and tends to be highly input-specific, as it all depends on the possible lengths and formats of the input. There’s thus no general rule that can be applied to all of these inputs, although it is often possible to show immediate warnings for inputs that can have: invalid characters (e.g. alphanumeric characters in a numeric input), impossible character combinations (e.g. Luhn validation), and too long inputs (e.g. a 15-digit phone number), can typically be warned against immediately (i.e. before the user leaves the field).
If resources are constrained, it’s best to “play it safe” and check input validity when the user leaves the field instead of half-seriously dabbling with advanced early warnings. After all, if the logic for these early warnings isn’t flawless, your users can end up with premature validations, and despite all the hard work your live inline validations may end up doing more harm than good.
When an error has occurred and a live inline error message is shown to users, they will try to correct it immediately. However, 10% of sites that have inline validation, do not remove the error message live, as the user resolves the issue.
During testing, the subjects often didn’t click out of a field – instead they would look intensely at the error message as they made corrections (i.e. on a keystroke level), expecting the error to be removed as soon as the field contained the valid input. Not removing error messages live, as the error is corrected, can thus cause grave issues as users are likely to misinterpret their newly corrected (and now valid) input to still contain errors. This is particularly true for fields where the validity is interdependent, such as email and password confirmation fields.
It’s therefore key that when an error is invoked using live inline validation, the logic for re-checking the validity of the invalid field doesn’t just happen when the user leaves the field. Error messages must instead live-update on a keystroke level (i.e. not just
onblur), and disappear the moment the user reaches a potentially valid input.
The danger here is in removing the error message too late – effectively the opposite of premature inline validation where the danger is in showing the error message too early!
To avoid the two pitfalls of inline validations it is thus important to give users a fair chance to enter a valid input before showing an error message (a conclusion we can often only draw once the user leaves the field), yet that same error message must be updated and removed instantly upon the user fixing their input (i.e. at each and every keystroke).
Lastly, during testing we observed positive inline validations to perform well. Instead of only telling users what they’ve done wrong, a site may also tell them when they’re doing something right. During testing, sites that used positive inline validation, by adding a checkmark next to valid input values, added a sense of delight and progression to the whole typing experience.
Positive inline validation is not only observed to add a sense of progression to an otherwise dull typing experience. It can also provide a better typing experience for risk-averse users trying to preempt validation errors, as well as users who otherwise tend to meticulously review the entire form before submitting it.
Our benchmark reveals that only 20% of major e-commerce sites with inline validation also use positive inline validation.
During the past 7 years of checkout testing we have consistently found inline validation – when implemented correctly – greatly improves the user’s error recovery experience. Luckily, this hasn’t gone unnoticed by the e-commerce industry at large, as we have observed a 4-fold increase in implementations during the last 4 years.
Yet, a high performing inline validation implementation requires close attention to details, due to the opposing logic for invoking and removing the live error messages:
This article presents the research findings from just 1 of the 850+ UX guidelines in Baymard Premium – get full access to learn how to create a “State of the Art” cart and checkout user experience.
Join 22,000+ readers and get Baymard’s research articles by RSS feed or
Topics include user experience, web design, and e-commerce
Articles are always delivered ad-free and in their full length
1-click unsubscribe at any time
Thanks for a great article.
I am wondering if you have any recommendations for dealing with prefilled fields. If the user enters a new form that has some fields already prefilled (e.g. from previously stored information), should those fields be already marked with a green check mark (positive validation) right after the form is shown? Or should they be marked only after the user enters and leaves the fields (on blur) even though the user doesn’t change their contents?
I think that the prefilled fields complicate this a bit, because they can also break the usual flow of how users fill out the form (i.e. field by field from top to bottom).
Hi Tom, yes pre-filled fields complicates it. We do have test findings on this in the full checkout study finding, but they are a bit to long for a comment here. In short: when loading a new page with pre-filled fields, these fields should generally be separated from any new empty fields. (if not a very high proportion of users will overlook the empty fields mixed into the prefilled fields).
‘the phone number wasn’t entered in the site’s desired format. Inline validation would have prevented both of the wasted error page reloads.’
And help text explaining the expected format could have eliminated the need for validation altogether, like ‘Please do not include your country code eg. (123-456-789)’ (though I would have coded it to accept both, then converted it to a singular format on the back end before storing in the database.
Yes how and when errors are shown to users (inline validation) is only a small part of optimizing a checkout flow.
We see in testing (as you also suggest) that proper labelling will reduce the issue (although not eliminate it), see some of our findings on field labeling and descriptions here: http://baymard.com/blog/checkout-form-field-descriptions
Also the error message itself is instrumental in how easy of difficult the error is to resolve, see:
Just to mention two.
Such a useful article, thank you.
There is a risk of the positive validation being misleading when used with free text fields such as name, postal address (manual entry) etc. When the focus leaves the field and the positive validation appears, the user could understandably assume that the data they’ve entered is correct, when there could be spelling mistakes present in their name or postal address.
The same could occur in an email address field when, in reality, it may only be the format that’s validated rather than the actual email address itself.
Do the benefits of using positive validation still outweigh those risks?
Good points Gill!
Typo under pitfall #1, “length” misspelled. Good article!
“Depending on the type of input, avoiding premature validation means that the validity of each field input should be checked when the user leaves the field or reaches the intended input lenght.”
I would also recommend to have the two email adresse fields on top of each other instead of next to each other. this makes it much easier to spot errors when the letters are represented in alignment.
You forgot to mention another annoying practice, live inline validation throwing up an error when my cursor is in the field but I switch away from the browser window—when I go to my password manager application to find the password! I haven’t entered anything yet but an error show up! Goes with the category “premature live inline validation”. AAAARRRGH!
On the American Eagle Outfitters’ Checkout UX error on phone number I believe the site should simply format the phone number instead of putting up an error because “wasn’t entered in the site’s desired format.” One of my pet peeves…
Third party payment gateways like stripe now provide asynchronous validation, so you do not have to empty the credit card fields if an error occurs, the error comes back right on page which is a great, great feature.
On another note, I do believe with mobile experience and the fact that people want to fill out forms the way and order they see fit (highly unpredictable), and the level of distraction inline live validation causes (especially when error messages appear to take space and make the page change in height), why not just validate when user commits? Scrolling up to the first error in page and let them decide how to fix it? I found this to be the least obtrusive way, but I have never tested it on real users.
Something I would like to add. Let’s say you have some fields all set to only display their error messages onblur. And you enter incorrect data in one and click in (or tab to) another. OK, the first will display an error message. Assuming you enter nothing in the input you are now in, it can be very annoying when you go back to correct the error, to get an error displayed for the second field. I believe this too needs to be addressed.
Thank you for this post. I didn’t even think about all this.
Some websites ask you to fill a very long form. During the filling, you copy everything to your excel of passwords and loose time.
At the end, when you click “send” you get a message “session expired” and you have to fill all the form again.
© 2021 Baymard Institute US: +1 (415) 315-9567 EU: +45 3696 9567 firstname.lastname@example.org