[14/n] PR85694: Rework overwidening detection

This patch is the main part of PR85694. The aim is to recognise at least: signed char *a, *b, *c; ... for (int i = 0; i < 2048; i++) c[i] = (a[i] + b[i]) >> 1; as an over-widening pattern, since the addition and shift can be done on shorts rather than ints. However, it ended up being a lot more general than that. The current over-widening pattern detection is limited to a few simple cases: logical ops with immediate second operands, and shifts by a constant. These cases are enough for common pixel-format conversion and can be detected in a peephole way. The loop above requires two generalisations of the current code: support for addition as well as logical ops, and support for non-constant second operands. These are harder to detect in the same peephole way, so the patch tries to take a more global approach. The idea is to get information about the minimum operation width in two ways: (1) by using the range information attached to the SSA_NAMEs (effectively a forward walk, since the range info is context-independent). (2) by back-propagating the number of output bits required by users of the result. As explained in the comments, there's a balance to be struck between narrowing an individual operation and fitting in with the surrounding code. The approach is pretty conservative: if we could narrow an operation to N bits without changing its semantics, it's OK to do that if: - no operations later in the chain require more than N bits; or - all internally-defined inputs are extended from N bits or fewer, and at least one of them is single-use. See the comments for the rationale. I didn't bother adding STMT_VINFO_* wrappers for the new fields since the code seemed more readable without. 2018-06-20 Richard Sandiford <richard.sandiford@arm.com> gcc/ * poly-int.h (print_hex): New function. * dumpfile.h (dump_dec, dump_hex): Declare. * dumpfile.c (dump_dec, dump_hex): New poly_wide_int functions. * tree-vectorizer.h (_stmt_vec_info): Add min_output_precision, min_input_precision, operation_precision and operation_sign. * tree-vect-patterns.c (vect_get_range_info): New function. (vect_same_loop_or_bb_p, vect_single_imm_use) (vect_operation_fits_smaller_type): Delete. (vect_look_through_possible_promotion): Add an optional single_use_p parameter. (vect_recog_over_widening_pattern): Rewrite to use new stmt_vec_info infomration. Handle one operation at a time. (vect_recog_cast_forwprop_pattern, vect_narrowable_type_p) (vect_truncatable_operation_p, vect_set_operation_type) (vect_set_min_input_precision): New functions. (vect_determine_min_output_precision_1): Likewise. (vect_determine_min_output_precision): Likewise. (vect_determine_precisions_from_range): Likewise. (vect_determine_precisions_from_users): Likewise. (vect_determine_stmt_precisions, vect_determine_precisions): Likewise. (vect_vect_recog_func_ptrs): Put over_widening first. Add cast_forwprop. (vect_pattern_recog): Call vect_determine_precisions. gcc/testsuite/ * gcc.dg/vect/vect-widen-mult-u8-u32.c: Check specifically for a widen_mult pattern. * gcc.dg/vect/vect-over-widen-1.c: Update the scan tests for new over-widening messages. * gcc.dg/vect/vect-over-widen-1-big-array.c: Likewise. * gcc.dg/vect/vect-over-widen-2.c: Likewise. * gcc.dg/vect/vect-over-widen-2-big-array.c: Likewise. * gcc.dg/vect/vect-over-widen-3.c: Likewise. * gcc.dg/vect/vect-over-widen-3-big-array.c: Likewise. * gcc.dg/vect/vect-over-widen-4.c: Likewise. * gcc.dg/vect/vect-over-widen-4-big-array.c: Likewise. * gcc.dg/vect/bb-slp-over-widen-1.c: New test. * gcc.dg/vect/bb-slp-over-widen-2.c: Likewise. * gcc.dg/vect/vect-over-widen-5.c: Likewise. * gcc.dg/vect/vect-over-widen-6.c: Likewise. * gcc.dg/vect/vect-over-widen-7.c: Likewise. * gcc.dg/vect/vect-over-widen-8.c: Likewise. * gcc.dg/vect/vect-over-widen-9.c: Likewise. * gcc.dg/vect/vect-over-widen-10.c: Likewise. * gcc.dg/vect/vect-over-widen-11.c: Likewise. * gcc.dg/vect/vect-over-widen-12.c: Likewise. * gcc.dg/vect/vect-over-widen-13.c: Likewise. * gcc.dg/vect/vect-over-widen-14.c: Likewise. * gcc.dg/vect/vect-over-widen-15.c: Likewise. * gcc.dg/vect/vect-over-widen-16.c: Likewise. * gcc.dg/vect/vect-over-widen-17.c: Likewise. * gcc.dg/vect/vect-over-widen-18.c: Likewise. * gcc.dg/vect/vect-over-widen-19.c: Likewise. * gcc.dg/vect/vect-over-widen-20.c: Likewise. * gcc.dg/vect/vect-over-widen-21.c: Likewise. From-SVN: r262333
2018-07-03 09:59:37 +00:00
parent 3239dde940
commit 370c2ebe8f
35 changed files with 1707 additions and 397 deletions
@@ -1,3 +1,29 @@
+2018-07-03  Richard Sandiford  <richard.sandiford@arm.com>
+
+	* poly-int.h (print_hex): New function.
+	* dumpfile.h (dump_dec, dump_hex): Declare.
+	* dumpfile.c (dump_dec, dump_hex): New poly_wide_int functions.
+	* tree-vectorizer.h (_stmt_vec_info): Add min_output_precision,
+	min_input_precision, operation_precision and operation_sign.
+	* tree-vect-patterns.c (vect_get_range_info): New function.
+	(vect_same_loop_or_bb_p, vect_single_imm_use)
+	(vect_operation_fits_smaller_type): Delete.
+	(vect_look_through_possible_promotion): Add an optional
+	single_use_p parameter.
+	(vect_recog_over_widening_pattern): Rewrite to use new
+	stmt_vec_info infomration.  Handle one operation at a time.
+	(vect_recog_cast_forwprop_pattern, vect_narrowable_type_p)
+	(vect_truncatable_operation_p, vect_set_operation_type)
+	(vect_set_min_input_precision): New functions.
+	(vect_determine_min_output_precision_1): Likewise.
+	(vect_determine_min_output_precision): Likewise.
+	(vect_determine_precisions_from_range): Likewise.
+	(vect_determine_precisions_from_users): Likewise.
+	(vect_determine_stmt_precisions, vect_determine_precisions): Likewise.
+	(vect_vect_recog_func_ptrs): Put over_widening first.
+	Add cast_forwprop.
+	(vect_pattern_recog): Call vect_determine_precisions.
+
 2018-07-03  Richard Sandiford  <richard.sandiford@arm.com>

 	* tree-vect-patterns.c (vect_mark_pattern_stmts): Remove pattern
@@ -633,6 +633,28 @@ template void dump_dec (dump_flags_t, const poly_uint64 &);
 template void dump_dec (dump_flags_t, const poly_offset_int &);
 template void dump_dec (dump_flags_t, const poly_widest_int &);

+void
+dump_dec (dump_flags_t dump_kind, const poly_wide_int &value, signop sgn)
+{
+  if (dump_file && (dump_kind & pflags))
+    print_dec (value, dump_file, sgn);
+
+  if (alt_dump_file && (dump_kind & alt_flags))
+    print_dec (value, alt_dump_file, sgn);
+}
+
+/* Output VALUE in hexadecimal to appropriate dump streams.  */
+
+void
+dump_hex (dump_flags_t dump_kind, const poly_wide_int &value)
+{
+  if (dump_file && (dump_kind & pflags))
+    print_hex (value, dump_file);
+
+  if (alt_dump_file && (dump_kind & alt_flags))
+    print_hex (value, alt_dump_file);
+}
+
 /* The current dump scope-nesting depth.  */

 static int dump_scope_depth;
@@ -439,6 +439,8 @@ extern bool enable_rtl_dump_file (void);

 template<unsigned int N, typename C>
 void dump_dec (dump_flags_t, const poly_int<N, C> &);
+extern void dump_dec (dump_flags_t, const poly_wide_int &, signop);
+extern void dump_hex (dump_flags_t, const poly_wide_int &);

 /* In tree-dump.c  */
 extern void dump_node (const_tree, dump_flags_t, FILE *);
@@ -2420,6 +2420,25 @@ print_dec (const poly_int_pod<N, C> &value, FILE *file)
 	     poly_coeff_traits<C>::signedness ? SIGNED : UNSIGNED);
 }

+/* Use print_hex to print VALUE to FILE.  */
+
+template<unsigned int N, typename C>
+void
+print_hex (const poly_int_pod<N, C> &value, FILE *file)
+{
+  if (value.is_constant ())
+    print_hex (value.coeffs[0], file);
+  else
+    {
+      fprintf (file, "[");
+      for (unsigned int i = 0; i < N; ++i)
+	{
+	  print_hex (value.coeffs[i], file);
+	  fputc (i == N - 1 ? ']' : ',', file);
+	}
+    }
+}
+
 /* Helper for calculating the distance between two points P1 and P2,
   in cases where known_le (P1, P2).  T1 and T2 are the types of the
   two positions, in either order.  The coefficients of P2 - P1 have
@@ -1,3 +1,36 @@
+2018-07-03  Richard Sandiford  <richard.sandiford@arm.com>
+
+	* gcc.dg/vect/vect-widen-mult-u8-u32.c: Check specifically for a
+	widen_mult pattern.
+	* gcc.dg/vect/vect-over-widen-1.c: Update the scan tests for new
+	over-widening messages.
+	* gcc.dg/vect/vect-over-widen-1-big-array.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-2.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-2-big-array.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-3.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-3-big-array.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-4.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-4-big-array.c: Likewise.
+	* gcc.dg/vect/bb-slp-over-widen-1.c: New test.
+	* gcc.dg/vect/bb-slp-over-widen-2.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-5.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-6.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-7.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-8.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-9.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-10.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-11.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-12.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-13.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-14.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-15.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-16.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-17.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-18.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-19.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-20.c: Likewise.
+	* gcc.dg/vect/vect-over-widen-21.c: Likewise.
+
 2018-07-03  Richard Sandiford  <richard.sandiford@arm.com>

 	* gcc.dg/vect/vect-mixed-size-cond-1.c: New test.
@@ -0,0 +1,66 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+/* Deliberate use of signed >>.  */
+#define DEF_LOOP(SIGNEDNESS)			\
+  void __attribute__ ((noipa))			\
+  f_##SIGNEDNESS (SIGNEDNESS char *restrict a,	\
+		  SIGNEDNESS char *restrict b,	\
+		  SIGNEDNESS char *restrict c)	\
+  {						\
+    a[0] = (b[0] + c[0]) >> 1;			\
+    a[1] = (b[1] + c[1]) >> 1;			\
+    a[2] = (b[2] + c[2]) >> 1;			\
+    a[3] = (b[3] + c[3]) >> 1;			\
+    a[4] = (b[4] + c[4]) >> 1;			\
+    a[5] = (b[5] + c[5]) >> 1;			\
+    a[6] = (b[6] + c[6]) >> 1;			\
+    a[7] = (b[7] + c[7]) >> 1;			\
+    a[8] = (b[8] + c[8]) >> 1;			\
+    a[9] = (b[9] + c[9]) >> 1;			\
+    a[10] = (b[10] + c[10]) >> 1;		\
+    a[11] = (b[11] + c[11]) >> 1;		\
+    a[12] = (b[12] + c[12]) >> 1;		\
+    a[13] = (b[13] + c[13]) >> 1;		\
+    a[14] = (b[14] + c[14]) >> 1;		\
+    a[15] = (b[15] + c[15]) >> 1;		\
+  }
+
+DEF_LOOP (signed)
+DEF_LOOP (unsigned)
+
+#define N 16
+
+#define TEST_LOOP(SIGNEDNESS, BASE_B, BASE_C)		\
+  {							\
+    SIGNEDNESS char a[N], b[N], c[N];			\
+    for (int i = 0; i < N; ++i)				\
+      {							\
+	b[i] = BASE_B + i * 15;				\
+	c[i] = BASE_C + i * 14;				\
+	asm volatile ("" ::: "memory");			\
+      }							\
+    f_##SIGNEDNESS (a, b, c);				\
+    for (int i = 0; i < N; ++i)				\
+      if (a[i] != (BASE_B + BASE_C + i * 29) >> 1)	\
+	__builtin_abort ();				\
+  }
+
+int
+main (void)
+{
+  check_vect ();
+
+  TEST_LOOP (signed, -128, -120);
+  TEST_LOOP (unsigned, 4, 10);
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump "demoting int to signed short" "slp2" { target { ! vect_widen_shift } } } } */
+/* { dg-final { scan-tree-dump "demoting int to unsigned short" "slp2" { target { ! vect_widen_shift } } } } */
+/* { dg-final { scan-tree-dump-times "basic block vectorized" 2 "vect" } } */
@@ -0,0 +1,65 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+/* Deliberate use of signed >>.  */
+#define DEF_LOOP(SIGNEDNESS)			\
+  void __attribute__ ((noipa))			\
+  f_##SIGNEDNESS (SIGNEDNESS char *restrict a,	\
+		  SIGNEDNESS char *restrict b,	\
+		  SIGNEDNESS char c)		\
+  {						\
+    a[0] = (b[0] + c) >> 1;			\
+    a[1] = (b[1] + c) >> 1;			\
+    a[2] = (b[2] + c) >> 1;			\
+    a[3] = (b[3] + c) >> 1;			\
+    a[4] = (b[4] + c) >> 1;			\
+    a[5] = (b[5] + c) >> 1;			\
+    a[6] = (b[6] + c) >> 1;			\
+    a[7] = (b[7] + c) >> 1;			\
+    a[8] = (b[8] + c) >> 1;			\
+    a[9] = (b[9] + c) >> 1;			\
+    a[10] = (b[10] + c) >> 1;			\
+    a[11] = (b[11] + c) >> 1;			\
+    a[12] = (b[12] + c) >> 1;			\
+    a[13] = (b[13] + c) >> 1;			\
+    a[14] = (b[14] + c) >> 1;			\
+    a[15] = (b[15] + c) >> 1;			\
+  }
+
+DEF_LOOP (signed)
+DEF_LOOP (unsigned)
+
+#define N 16
+
+#define TEST_LOOP(SIGNEDNESS, BASE_B, C)		\
+  {							\
+    SIGNEDNESS char a[N], b[N], c[N];			\
+    for (int i = 0; i < N; ++i)				\
+      {							\
+	b[i] = BASE_B + i * 15;				\
+	asm volatile ("" ::: "memory");			\
+      }							\
+    f_##SIGNEDNESS (a, b, C);				\
+    for (int i = 0; i < N; ++i)				\
+      if (a[i] != (BASE_B + C + i * 15) >> 1)		\
+	__builtin_abort ();				\
+  }
+
+int
+main (void)
+{
+  check_vect ();
+
+  TEST_LOOP (signed, -128, -120);
+  TEST_LOOP (unsigned, 4, 250);
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump "demoting int to signed short" "slp2" { target { ! vect_widen_shift } } } } */
+/* { dg-final { scan-tree-dump "demoting int to unsigned short" "slp2" { target { ! vect_widen_shift } } } } */
+/* { dg-final { scan-tree-dump-times "basic block vectorized" 2 "vect" } } */
@@ -58,7 +58,9 @@ int main (void)
 }

 /* { dg-final { scan-tree-dump-times "vect_recog_widen_shift_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 4 "vect" { target { ! vect_widen_shift } } } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 5} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -62,8 +62,9 @@ int main (void)
 }

 /* { dg-final { scan-tree-dump-times "vect_recog_widen_shift_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 4 "vect" { target { { ! vect_sizes_32B_16B } && { ! vect_widen_shift } } } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 8 "vect" { target vect_sizes_32B_16B } } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 5} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -0,0 +1,19 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS unsigned
+#define BASE_B 4
+#define BASE_C 40
+#endif
+
+#include "vect-over-widen-9.c"
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,63 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS signed
+#define BASE_B -128
+#define BASE_C -100
+#endif
+
+#define N 50
+
+/* Both range analysis and backward propagation from the truncation show
+   that these calculations can be done in SIGNEDNESS short, with "res"
+   being extended for the store to d[i].  */
+void __attribute__ ((noipa))
+f (SIGNEDNESS char *restrict a, SIGNEDNESS char *restrict b,
+   SIGNEDNESS char *restrict c, int *restrict d)
+{
+  for (int i = 0; i < N; ++i)
+    {
+      /* Deliberate use of signed >>.  */
+      int res = b[i] + c[i];
+      a[i] = (res + (res >> 1)) >> 2;
+      d[i] = res;
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  SIGNEDNESS char a[N], b[N], c[N];
+  int d[N];
+  for (int i = 0; i < N; ++i)
+    {
+      b[i] = BASE_B + i * 5;
+      c[i] = BASE_C + i * 4;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c, d);
+  for (int i = 0; i < N; ++i)
+    {
+      int res = BASE_B + BASE_C + i * 9;
+      if (a[i] != ((res + (res >> 1)) >> 2))
+	__builtin_abort ();
+      if (d[i] != res)
+	__builtin_abort ();
+    }
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */
+/* { dg-final { scan-tree-dump {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,19 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS unsigned
+#define BASE_B 4
+#define BASE_C 40
+#endif
+
+#include "vect-over-widen-11.c"
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */
+/* { dg-final { scan-tree-dump {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,50 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS signed
+#define BASE_B -128
+#define BASE_C -120
+#endif
+
+#define N 50
+
+/* We rely on range analysis to show that these calculations can be done
+   in SIGNEDNESS short.  */
+void __attribute__ ((noipa))
+f (SIGNEDNESS char *restrict a, SIGNEDNESS char *restrict b,
+   SIGNEDNESS char *restrict c)
+{
+  for (int i = 0; i < N; ++i)
+    a[i] = (b[i] + c[i]) / 2;
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  SIGNEDNESS char a[N], b[N], c[N];
+  for (int i = 0; i < N; ++i)
+    {
+      b[i] = BASE_B + i * 5;
+      c[i] = BASE_C + i * 4;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != (BASE_B + BASE_C + i * 9) / 2)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* / 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* = \(signed char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,18 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS unsigned
+#define BASE_B 4
+#define BASE_C 40
+#endif
+
+#include "vect-over-widen-13.c"
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* = \(unsigned char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,52 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS signed
+#define BASE_B -128
+#define BASE_C -120
+#endif
+
+#define N 50
+
+/* We rely on range analysis to show that these calculations can be done
+   in SIGNEDNESS short, with the result being extended to int for the
+   store.  */
+void __attribute__ ((noipa))
+f (int *restrict a, SIGNEDNESS char *restrict b,
+   SIGNEDNESS char *restrict c)
+{
+  for (int i = 0; i < N; ++i)
+    a[i] = (b[i] + c[i]) / 2;
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  int a[N];
+  SIGNEDNESS char b[N], c[N];
+  for (int i = 0; i < N; ++i)
+    {
+      b[i] = BASE_B + i * 5;
+      c[i] = BASE_C + i * 4;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != (BASE_B + BASE_C + i * 9) / 2)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* / 2} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vect_recog_cast_forwprop_pattern: detected} "vect" } } */
+/* { dg-final { scan-tree-dump {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,18 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS unsigned
+#define BASE_B 4
+#define BASE_C 40
+#endif
+
+#include "vect-over-widen-15.c"
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vect_recog_cast_forwprop_pattern: detected} "vect" } } */
+/* { dg-final { scan-tree-dump {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,46 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#define N 1024
+
+/* This should not be treated as an over-widening pattern, even though
+   "(b[i] & 0xef) | 0x80)" could be done in unsigned chars.  */
+
+void __attribute__ ((noipa))
+f (unsigned short *restrict a, unsigned short *restrict b)
+{
+  for (__INTPTR_TYPE__ i = 0; i < N; ++i)
+    {
+      unsigned short foo = ((b[i] & 0xef) | 0x80) + (a[i] << 4);
+      a[i] = foo;
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  unsigned short a[N], b[N];
+  for (int i = 0; i < N; ++i)
+    {
+      a[i] = i;
+      b[i] = i * 3;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != ((((i * 3) & 0xef) | 0x80) + (i << 4)))
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump-not {vect_recog_over_widening_pattern: detected} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^\n]*char} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,50 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#define N 1024
+
+/* This should be treated as an over-widening pattern: we can truncate
+   b to unsigned char after loading it and do all the computation in
+   unsigned char.  */
+
+void __attribute__ ((noipa))
+f (unsigned char *restrict a, unsigned short *restrict b)
+{
+  for (__INTPTR_TYPE__ i = 0; i < N; ++i)
+    {
+      unsigned short foo = ((b[i] & 0xef) | 0x80) + (a[i] << 4);
+      a[i] = foo;
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  unsigned char a[N];
+  unsigned short b[N];
+  for (int i = 0; i < N; ++i)
+    {
+      a[i] = i;
+      b[i] = i * 3;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != (unsigned char) ((((i * 3) & 0xef) | 0x80) + (i << 4)))
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* &} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* |} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* <<} "vect" } } */
+/* { dg-final { scan-tree-dump {vector[^\n]*char} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,53 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#define N 111
+
+/* This shouldn't be treated as an over-widening operation: it's better
+   to reuse the extensions of di and ei for di + ei than to add them
+   as shorts and introduce a third extension.  */
+
+void __attribute__ ((noipa))
+f (unsigned int *restrict a, unsigned int *restrict b,
+   unsigned int *restrict c, unsigned char *restrict d,
+   unsigned char *restrict e)
+{
+  for (__INTPTR_TYPE__ i = 0; i < N; ++i)
+    {
+      unsigned int di = d[i];
+      unsigned int ei = e[i];
+      a[i] = di;
+      b[i] = ei;
+      c[i] = di + ei;
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  unsigned int a[N], b[N], c[N];
+  unsigned char d[N], e[N];
+  for (int i = 0; i < N; ++i)
+    {
+      d[i] = i * 2 + 3;
+      e[i] = i + 100;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c, d, e);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != i * 2 + 3
+	|| b[i] != i + 100
+	|| c[i] != i * 3 + 103)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump-not {vect_recog_over_widening_pattern: detected} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -57,7 +57,12 @@ int main (void)
  return 0;
 }

-/* Final value stays in int, so no over-widening is detected at the moment.  */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 0 "vect" } } */
+/* This is an over-widening even though the final result is still an int.
+   It's better to do one vector of ops on chars and then widen than to
+   widen and then do 4 vectors of ops on ints.  */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 5} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -57,7 +57,12 @@ int main (void)
  return 0;
 }

-/* Final value stays in int, so no over-widening is detected at the moment.  */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 0 "vect" } } */
+/* This is an over-widening even though the final result is still an int.
+   It's better to do one vector of ops on chars and then widen than to
+   widen and then do 4 vectors of ops on ints.  */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 5} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -0,0 +1,53 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#define N 111
+
+/* This shouldn't be treated as an over-widening operation: it's better
+   to reuse the extensions of di and ei for di + ei than to add them
+   as shorts and introduce a third extension.  */
+
+void __attribute__ ((noipa))
+f (unsigned int *restrict a, unsigned int *restrict b,
+   unsigned int *restrict c, unsigned char *restrict d,
+   unsigned char *restrict e)
+{
+  for (__INTPTR_TYPE__ i = 0; i < N; ++i)
+    {
+      int di = d[i];
+      int ei = e[i];
+      a[i] = di;
+      b[i] = ei;
+      c[i] = di + ei;
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  unsigned int a[N], b[N], c[N];
+  unsigned char d[N], e[N];
+  for (int i = 0; i < N; ++i)
+    {
+      d[i] = i * 2 + 3;
+      e[i] = i + 100;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c, d, e);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != i * 2 + 3
+	|| b[i] != i + 100
+	|| c[i] != i * 3 + 103)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump-not {vect_recog_over_widening_pattern: detected} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,51 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#define N 111
+
+/* This shouldn't be treated as an over-widening operation: it's better
+   to reuse the extensions of di and ei for di + ei than to add them
+   as shorts and introduce a third extension.  */
+
+void __attribute__ ((noipa))
+f (unsigned int *restrict a, unsigned int *restrict b,
+   unsigned int *restrict c, unsigned char *restrict d,
+   unsigned char *restrict e)
+{
+  for (__INTPTR_TYPE__ i = 0; i < N; ++i)
+    {
+      a[i] = d[i];
+      b[i] = e[i];
+      c[i] = d[i] + e[i];
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  unsigned int a[N], b[N], c[N];
+  unsigned char d[N], e[N];
+  for (int i = 0; i < N; ++i)
+    {
+      d[i] = i * 2 + 3;
+      e[i] = i + 100;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c, d, e);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != i * 2 + 3
+	|| b[i] != i + 100
+	|| c[i] != i * 3 + 103)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump-not {vect_recog_over_widening_pattern: detected} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -59,7 +59,9 @@ int main (void)
  return 0;
 }

-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 2 "vect" { target { ! vect_widen_shift } } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 1 "vect" { target vect_widen_shift } } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 9} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -57,6 +57,9 @@ int main (void)
  return 0;
 }

-/* { dg-final { scan-tree-dump "vect_recog_over_widening_pattern: detected" "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 9} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -62,7 +62,9 @@ int main (void)
 }

 /* { dg-final { scan-tree-dump-times "vect_recog_widen_shift_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 4 "vect" { target { ! vect_widen_shift } } } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 5} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -66,8 +66,9 @@ int main (void)
 }

 /* { dg-final { scan-tree-dump-times "vect_recog_widen_shift_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 2 "vect" { target vect_widen_shift } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 4 "vect" { target { { ! vect_sizes_32B_16B } && { ! vect_widen_shift } } } } } */
-/* { dg-final { scan-tree-dump-times "vect_recog_over_widening_pattern: detected" 8 "vect" { target vect_sizes_32B_16B } } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 3} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* << 8} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 5} "vect" } } */
 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" } } */

@@ -0,0 +1,51 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS signed
+#define BASE_B -128
+#define BASE_C -100
+#endif
+
+#define N 50
+
+/* Both range analysis and backward propagation from the truncation show
+   that these calculations can be done in SIGNEDNESS short.  */
+void __attribute__ ((noipa))
+f (SIGNEDNESS char *restrict a, SIGNEDNESS char *restrict b,
+   SIGNEDNESS char *restrict c)
+{
+  /* Deliberate use of signed >>.  */
+  for (int i = 0; i < N; ++i)
+    a[i] = (b[i] + c[i]) >> 1;
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  SIGNEDNESS char a[N], b[N], c[N];
+  for (int i = 0; i < N; ++i)
+    {
+      b[i] = BASE_B + i * 5;
+      c[i] = BASE_C + i * 4;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != (BASE_B + BASE_C + i * 9) >> 1)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,16 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#define SIGNEDNESS unsigned
+#define BASE_B 4
+#define BASE_C 40
+
+#include "vect-over-widen-5.c"
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,53 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS signed
+#define BASE_B -128
+#define BASE_C -100
+#define D -120
+#endif
+
+#define N 50
+
+/* Both range analysis and backward propagation from the truncation show
+   that these calculations can be done in SIGNEDNESS short.  */
+void __attribute__ ((noipa))
+f (SIGNEDNESS char *restrict a, SIGNEDNESS char *restrict b,
+   SIGNEDNESS char *restrict c, SIGNEDNESS char d)
+{
+  int promoted_d = d;
+  for (int i = 0; i < N; ++i)
+    /* Deliberate use of signed >>.  */
+    a[i] = (b[i] + c[i] + promoted_d) >> 2;
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  SIGNEDNESS char a[N], b[N], c[N];
+  for (int i = 0; i < N; ++i)
+    {
+      b[i] = BASE_B + i * 5;
+      c[i] = BASE_C + i * 4;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c, D);
+  for (int i = 0; i < N; ++i)
+    if (a[i] != (BASE_B + BASE_C + D + i * 9) >> 2)
+      __builtin_abort ();
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,19 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS unsigned
+#define BASE_B 4
+#define BASE_C 40
+#define D 251
+#endif
+
+#include "vect-over-widen-7.c"
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(unsigned char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -0,0 +1,58 @@
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_shift } */
+/* { dg-require-effective-target vect_pack_trunc } */
+/* { dg-require-effective-target vect_unpack } */
+
+#include "tree-vect.h"
+
+#ifndef SIGNEDNESS
+#define SIGNEDNESS signed
+#define BASE_B -128
+#define BASE_C -100
+#endif
+
+#define N 50
+
+/* Both range analysis and backward propagation from the truncation show
+   that these calculations can be done in SIGNEDNESS short.  */
+void __attribute__ ((noipa))
+f (SIGNEDNESS char *restrict a, SIGNEDNESS char *restrict b,
+   SIGNEDNESS char *restrict c)
+{
+  for (int i = 0; i < N; ++i)
+    {
+      /* Deliberate use of signed >>.  */
+      int res = b[i] + c[i];
+      a[i] = (res + (res >> 1)) >> 2;
+    }
+}
+
+int
+main (void)
+{
+  check_vect ();
+
+  SIGNEDNESS char a[N], b[N], c[N];
+  for (int i = 0; i < N; ++i)
+    {
+      b[i] = BASE_B + i * 5;
+      c[i] = BASE_C + i * 4;
+      asm volatile ("" ::: "memory");
+    }
+  f (a, b, c);
+  for (int i = 0; i < N; ++i)
+    {
+      int res = BASE_B + BASE_C + i * 9;
+      if (a[i] != ((res + (res >> 1)) >> 2))
+	__builtin_abort ();
+    }
+
+  return 0;
+}
+
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* \+ } "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 1} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_over_widening_pattern: detected:[^\n]* >> 2} "vect" } } */
+/* { dg-final { scan-tree-dump {vect_recog_cast_forwprop_pattern: detected:[^\n]* \(signed char\)} "vect" } } */
+/* { dg-final { scan-tree-dump-not {vector[^ ]* int} "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loop" 1 "vect" } } */
@@ -43,5 +43,5 @@ int main (void)

 /* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_widen_mult_qi_to_hi || vect_unpack } } } } */
 /* { dg-final { scan-tree-dump-times "vect_recog_widen_mult_pattern: detected" 1 "vect" { target vect_widen_mult_qi_to_hi_pattern } } } */
-/* { dg-final { scan-tree-dump-times "pattern recognized" 1 "vect" { target vect_widen_mult_qi_to_hi_pattern } } } */
+/* { dg-final { scan-tree-dump-times "widen_mult pattern recognized" 1 "vect" { target vect_widen_mult_qi_to_hi_pattern } } } */

@@ -899,6 +899,21 @@ typedef struct _stmt_vec_info {

  /* The number of scalar stmt references from active SLP instances.  */
  unsigned int num_slp_uses;
+
+  /* If nonzero, the lhs of the statement could be truncated to this
+     many bits without affecting any users of the result.  */
+  unsigned int min_output_precision;
+
+  /* If nonzero, all non-boolean input operands have the same precision,
+     and they could each be truncated to this many bits without changing
+     the result.  */
+  unsigned int min_input_precision;
+
+  /* If OPERATION_BITS is nonzero, the statement could be performed on
+     an integer with the sign and number of bits given by OPERATION_SIGN
+     and OPERATION_BITS without changing the result.  */
+  unsigned int operation_precision;
+  signop operation_sign;
 } *stmt_vec_info;

 /* Information about a gather/scatter call.  */