TCG code generator (c896fe29) | Commits | gwj / at91sam9263

Commit c896fe29d6c8ae6cde3917727812ced3f2e536a4

Authored by bellard 2008-02-01 10:05:41 +0000

TCG code generator


git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@3943 c046a42c-6fe2-441c-8c8c-71466251a162

Inline Side-by-side

Showing 13 changed files with 7026 additions and 0 deletions

tcg/LICENSE 0 → 100644

View file @c896fe2

		1	+All the files in this directory and subdirectories are released under
		2	+a BSD like license (see header in each file). No other license is
		3	+accepted.

tcg/README 0 → 100644

View file @c896fe2

		1	+Tiny Code Generator - Fabrice Bellard.
		2	+
		3	+1) Introduction
		4	+
		5	+TCG (Tiny Code Generator) began as a generic backend for a C
		6	+compiler. It was simplified to be used in QEMU. It also has its roots
		7	+in the QOP code generator written by Paul Brook.
		8	+
		9	+2) Definitions
		10	+
		11	+The TCG "target" is the architecture for which we generate the
		12	+code. It is of course not the same as the "target" of QEMU which is
		13	+the emulated architecture. As TCG started as a generic C backend used
		14	+for cross compiling, it is assumed that the TCG target is different
		15	+from the host, although it is never the case for QEMU.
		16	+
		17	+A TCG "function" corresponds to a QEMU Translated Block (TB).
		18	+
		19	+A TCG "temporary" is a variable only live in a given
		20	+function. Temporaries are allocated explicitely in each function.
		21	+
		22	+A TCG "global" is a variable which is live in all the functions. They
		23	+are defined before the functions defined. A TCG global can be a memory
		24	+location (e.g. a QEMU CPU register), a fixed host register (e.g. the
		25	+QEMU CPU state pointer) or a memory location which is stored in a
		26	+register outside QEMU TBs (not implemented yet).
		27	+
		28	+A TCG "basic block" corresponds to a list of instructions terminated
		29	+by a branch instruction.
		30	+
		31	+3) Intermediate representation
		32	+
		33	+3.1) Introduction
		34	+
		35	+TCG instructions operate on variables which are temporaries or
		36	+globals. TCG instructions and variables are strongly typed. Two types
		37	+are supported: 32 bit integers and 64 bit integers. Pointers are
		38	+defined as an alias to 32 bit or 64 bit integers depending on the TCG
		39	+target word size.
		40	+
		41	+Each instruction has a fixed number of output variable operands, input
		42	+variable operands and always constant operands.
		43	+
		44	+The notable exception is the call instruction which has a variable
		45	+number of outputs and inputs.
		46	+
		47	+In the textual form, output operands come first, followed by input
		48	+operands, followed by constant operands. The output type is included
		49	+in the instruction name. Constants are prefixed with a '$'.
		50	+
		51	+add_i32 t0, t1, t2 (t0 <- t1 + t2)
		52	+
		53	+sub_i64 t2, t3, $4 (t2 <- t3 - 4)
		54	+
		55	+3.2) Assumptions
		56	+
		57	+* Basic blocks
		58	+
		59	+- Basic blocks end after branches (e.g. brcond_i32 instruction),
		60	+ goto_tb and exit_tb instructions.
		61	+- Basic blocks end before legacy dyngen operations.
		62	+- Basic blocks start after the end of a previous basic block, at a
		63	+ set_label instruction or after a legacy dyngen operation.
		64	+
		65	+After the end of a basic block, temporaries at destroyed and globals
		66	+are stored at their initial storage (register or memory place
		67	+depending on their declarations).
		68	+
		69	+* Floating point types are not supported yet
		70	+
		71	+* Pointers: depending on the TCG target, pointer size is 32 bit or 64
		72	+ bit. The type TCG_TYPE_PTR is an alias to TCG_TYPE_I32 or
		73	+ TCG_TYPE_I64.
		74	+
		75	+* Helpers:
		76	+
		77	+Using the tcg_gen_helper_x_y it is possible to call any function
		78	+taking i32, i64 or pointer types types. Before calling an helper, all
		79	+globals are stored at their canonical location and it is assumed that
		80	+the function can modify them. In the future, function modifiers will
		81	+be allowed to tell that the helper does not read or write some globals.
		82	+
		83	+On some TCG targets (e.g. x86), several calling conventions are
		84	+supported.
		85	+
		86	+* Branches:
		87	+
		88	+Use the instruction 'br' to jump to a label. Use 'jmp' to jump to an
		89	+explicit address. Conditional branches can only jump to labels.
		90	+
		91	+3.3) Code Optimizations
		92	+
		93	+When generating instructions, you can count on at least the following
		94	+optimizations:
		95	+
		96	+- Single instructions are simplified, e.g.
		97	+
		98	+ and_i32 t0, t0, $0xffffffff
		99	+
		100	+ is suppressed.
		101	+
		102	+- A liveness analysis is done at the basic block level. The
		103	+ information is used to suppress moves from a dead temporary to
		104	+ another one. It is also used to remove instructions which compute
		105	+ dead results. The later is especially useful for condition code
		106	+ optimisation in QEMU.
		107	+
		108	+ In the following example:
		109	+
		110	+ add_i32 t0, t1, t2
		111	+ add_i32 t0, t0, $1
		112	+ mov_i32 t0, $1
		113	+
		114	+ only the last instruction is kept.
		115	+
		116	+- A macro system is supported (may get closer to function inlining
		117	+ some day). It is useful if the liveness analysis is likely to prove
		118	+ that some results of a computation are indeed not useful. With the
		119	+ macro system, the user can provide several alternative
		120	+ implementations which are used depending on the used results. It is
		121	+ especially useful for condition code optimisation in QEMU.
		122	+
		123	+ Here is an example:
		124	+
		125	+ macro_2 t0, t1, $1
		126	+ mov_i32 t0, $0x1234
		127	+
		128	+ The macro identified by the ID "$1" normally returns the values t0
		129	+ and t1. Suppose its implementation is:
		130	+
		131	+ macro_start
		132	+ brcond_i32 t2, $0, $TCG_COND_EQ, $1
		133	+ mov_i32 t0, $2
		134	+ br $2
		135	+ set_label $1
		136	+ mov_i32 t0, $3
		137	+ set_label $2
		138	+ add_i32 t1, t3, t4
		139	+ macro_end
		140	+
		141	+ If t0 is not used after the macro, the user can provide a simpler
		142	+ implementation:
		143	+
		144	+ macro_start
		145	+ add_i32 t1, t2, t4
		146	+ macro_end
		147	+
		148	+ TCG automatically chooses the right implementation depending on
		149	+ which macro outputs are used after it.
		150	+
		151	+ Note that if TCG did more expensive optimizations, macros would be
		152	+ less useful. In the previous example a macro is useful because the
		153	+ liveness analysis is done on each basic block separately. Hence TCG
		154	+ cannot remove the code computing 't0' even if it is not used after
		155	+ the first macro implementation.
		156	+
		157	+3.4) Instruction Reference
		158	+
		159	+********* Function call
		160	+
		161	+* call <ret> <params> ptr
		162	+
		163	+call function 'ptr' (pointer type)
		164	+
		165	+<ret> optional 32 bit or 64 bit return value
		166	+<params> optional 32 bit or 64 bit parameters
		167	+
		168	+********* Jumps/Labels
		169	+
		170	+* jmp t0
		171	+
		172	+Absolute jump to address t0 (pointer type).
		173	+
		174	+* set_label $label
		175	+
		176	+Define label 'label' at the current program point.
		177	+
		178	+* br $label
		179	+
		180	+Jump to label.
		181	+
		182	+* brcond_i32/i64 cond, t0, t1, label
		183	+
		184	+Conditional jump if t0 cond t1 is true. cond can be:
		185	+ TCG_COND_EQ
		186	+ TCG_COND_NE
		187	+ TCG_COND_LT /* signed */
		188	+ TCG_COND_GE /* signed */
		189	+ TCG_COND_LE /* signed */
		190	+ TCG_COND_GT /* signed */
		191	+ TCG_COND_LTU /* unsigned */
		192	+ TCG_COND_GEU /* unsigned */
		193	+ TCG_COND_LEU /* unsigned */
		194	+ TCG_COND_GTU /* unsigned */
		195	+
		196	+********* Arithmetic
		197	+
		198	+* add_i32/i64 t0, t1, t2
		199	+
		200	+t0=t1+t2
		201	+
		202	+* sub_i32/i64 t0, t1, t2
		203	+
		204	+t0=t1-t2
		205	+
		206	+* mul_i32/i64 t0, t1, t2
		207	+
		208	+t0=t1*t2
		209	+
		210	+* div_i32/i64 t0, t1, t2
		211	+
		212	+t0=t1/t2 (signed). Undefined behavior if division by zero or overflow.
		213	+
		214	+* divu_i32/i64 t0, t1, t2
		215	+
		216	+t0=t1/t2 (unsigned). Undefined behavior if division by zero.
		217	+
		218	+* rem_i32/i64 t0, t1, t2
		219	+
		220	+t0=t1%t2 (signed). Undefined behavior if division by zero or overflow.
		221	+
		222	+* remu_i32/i64 t0, t1, t2
		223	+
		224	+t0=t1%t2 (unsigned). Undefined behavior if division by zero.
		225	+
		226	+* and_i32/i64 t0, t1, t2
		227	+
		228	+********* Logical
		229	+
		230	+t0=t1&t2
		231	+
		232	+* or_i32/i64 t0, t1, t2
		233	+
		234	+t0=t1\|t2
		235	+
		236	+* xor_i32/i64 t0, t1, t2
		237	+
		238	+t0=t1^t2
		239	+
		240	+* shl_i32/i64 t0, t1, t2
		241	+
		242	+********* Shifts
		243	+
		244	+* shl_i32/i64 t0, t1, t2
		245	+
		246	+t0=t1 << t2. Undefined behavior if t2 < 0 or t2 >= 32 (resp 64)
		247	+
		248	+* shr_i32/i64 t0, t1, t2
		249	+
		250	+t0=t1 >> t2 (unsigned). Undefined behavior if t2 < 0 or t2 >= 32 (resp 64)
		251	+
		252	+* sar_i32/i64 t0, t1, t2
		253	+
		254	+t0=t1 >> t2 (signed). Undefined behavior if t2 < 0 or t2 >= 32 (resp 64)
		255	+
		256	+********* Misc
		257	+
		258	+* mov_i32/i64 t0, t1
		259	+
		260	+t0 = t1
		261	+
		262	+Move t1 to t0 (both operands must have the same type).
		263	+
		264	+* ext8s_i32/i64 t0, t1
		265	+ext16s_i32/i64 t0, t1
		266	+ext32s_i64 t0, t1
		267	+
		268	+8, 16 or 32 bit sign extension (both operands must have the same type)
		269	+
		270	+* bswap16_i32 t0, t1
		271	+
		272	+16 bit byte swap on a 32 bit value. The two high order bytes must be set
		273	+to zero.
		274	+
		275	+* bswap_i32 t0, t1
		276	+
		277	+32 bit byte swap
		278	+
		279	+* bswap_i64 t0, t1
		280	+
		281	+64 bit byte swap
		282	+
		283	+********* Type conversions
		284	+
		285	+* ext_i32_i64 t0, t1
		286	+Convert t1 (32 bit) to t0 (64 bit) and does sign extension
		287	+
		288	+* extu_i32_i64 t0, t1
		289	+Convert t1 (32 bit) to t0 (64 bit) and does zero extension
		290	+
		291	+* trunc_i64_i32 t0, t1
		292	+Truncate t1 (64 bit) to t0 (32 bit)
		293	+
		294	+********* Load/Store
		295	+
		296	+* ld_i32/i64 t0, t1, offset
		297	+ld8s_i32/i64 t0, t1, offset
		298	+ld8u_i32/i64 t0, t1, offset
		299	+ld16s_i32/i64 t0, t1, offset
		300	+ld16u_i32/i64 t0, t1, offset
		301	+ld32s_i64 t0, t1, offset
		302	+ld32u_i64 t0, t1, offset
		303	+
		304	+t0 = read(t1 + offset)
		305	+Load 8, 16, 32 or 64 bits with or without sign extension from host memory.
		306	+offset must be a constant.
		307	+
		308	+* st_i32/i64 t0, t1, offset
		309	+st8_i32/i64 t0, t1, offset
		310	+st16_i32/i64 t0, t1, offset
		311	+st32_i64 t0, t1, offset
		312	+
		313	+write(t0, t1 + offset)
		314	+Write 8, 16, 32 or 64 bits to host memory.
		315	+
		316	+********* QEMU specific operations
		317	+
		318	+* tb_exit t0
		319	+
		320	+Exit the current TB and return the value t0 (word type).
		321	+
		322	+* goto_tb index
		323	+
		324	+Exit the current TB and jump to the TB index 'index' (constant) if the
		325	+current TB was linked to this TB. Otherwise execute the next
		326	+instructions.
		327	+
		328	+* qemu_ld_i32/i64 t0, t1, flags
		329	+qemu_ld8u_i32/i64 t0, t1, flags
		330	+qemu_ld8s_i32/i64 t0, t1, flags
		331	+qemu_ld16u_i32/i64 t0, t1, flags
		332	+qemu_ld16s_i32/i64 t0, t1, flags
		333	+qemu_ld32u_i64 t0, t1, flags
		334	+qemu_ld32s_i64 t0, t1, flags
		335	+
		336	+Load data at the QEMU CPU address t1 into t0. t1 has the QEMU CPU
		337	+address type. 'flags' contains the QEMU memory index (selects user or
		338	+kernel access) for example.
		339	+
		340	+* qemu_st_i32/i64 t0, t1, flags
		341	+qemu_st8_i32/i64 t0, t1, flags
		342	+qemu_st16_i32/i64 t0, t1, flags
		343	+qemu_st32_i64 t0, t1, flags
		344	+
		345	+Store the data t0 at the QEMU CPU Address t1. t1 has the QEMU CPU
		346	+address type. 'flags' contains the QEMU memory index (selects user or
		347	+kernel access) for example.
		348	+
		349	+Note 1: Some shortcuts are defined when the last operand is known to be
		350	+a constant (e.g. addi for add, movi for mov).
		351	+
		352	+Note 2: When using TCG, the opcodes must never be generated directly
		353	+as some of them may not be available as "real" opcodes. Always use the
		354	+function tcg_gen_xxx(args).
		355	+
		356	+4) Backend
		357	+
		358	+tcg-target.h contains the target specific definitions. tcg-target.c
		359	+contains the target specific code.
		360	+
		361	+4.1) Assumptions
		362	+
		363	+The target word size (TCG_TARGET_REG_BITS) is expected to be 32 bit or
		364	+64 bit. It is expected that the pointer has the same size as the word.
		365	+
		366	+On a 32 bit target, all 64 bit operations are converted to 32 bits. A
		367	+few specific operations must be implemented to allow it (see add2_i32,
		368	+sub2_i32, brcond2_i32).
		369	+
		370	+Floating point operations are not supported in this version. A
		371	+previous incarnation of the code generator had full support of them,
		372	+but it is better to concentrate on integer operations first.
		373	+
		374	+On a 64 bit target, no assumption is made in TCG about the storage of
		375	+the 32 bit values in 64 bit registers.
		376	+
		377	+4.2) Constraints
		378	+
		379	+GCC like constraints are used to define the constraints of every
		380	+instruction. Memory constraints are not supported in this
		381	+version. Aliases are specified in the input operands as for GCC.
		382	+
		383	+A target can define specific register or constant constraints. If an
		384	+operation uses a constant input constraint which does not allow all
		385	+constants, it must also accept registers in order to have a fallback.
		386	+
		387	+The movi_i32 and movi_i64 operations must accept any constants.
		388	+
		389	+The mov_i32 and mov_i64 operations must accept any registers of the
		390	+same type.
		391	+
		392	+The ld/st instructions must accept signed 32 bit constant offsets. It
		393	+can be implemented by reserving a specific register to compute the
		394	+address if the offset is too big.
		395	+
		396	+The ld/st instructions must accept any destination (ld) or source (st)
		397	+register.
		398	+
		399	+4.3) Function call assumptions
		400	+
		401	+- The only supported types for parameters and return value are: 32 and
		402	+ 64 bit integers and pointer.
		403	+- The stack grows downwards.
		404	+- The first N parameters are passed in registers.
		405	+- The next parameters are passed on the stack by storing them as words.
		406	+- Some registers are clobbered during the call.
		407	+- The function can return 0 or 1 value in registers. On a 32 bit
		408	+ target, functions must be able to return 2 values in registers for
		409	+ 64 bit return type.
		410	+
		411	+5) Migration from dyngen to TCG
		412	+
		413	+TCG is backward compatible with QEMU "dyngen" operations. It means
		414	+that TCG instructions can be freely mixed with dyngen operations. It
		415	+is expected that QEMU targets will be progressively fully converted to
		416	+TCG. Once a target is fully converted to dyngen, it will be possible
		417	+to apply more optimizations because more registers will be free for
		418	+the generated code.
		419	+
		420	+The exception model is the same as the dyngen one.

tcg/TODO 0 → 100644

View file @c896fe2

		1	+- test macro system
		2	+
		3	+- test conditional jumps
		4	+
		5	+- test mul, div, ext8s, ext16s, bswap
		6	+
		7	+- generate a global TB prologue and epilogue to save/restore registers
		8	+ to/from the CPU state and to reserve a stack frame to optimize
		9	+ helper calls. Modify cpu-exec.c so that it does not use global
		10	+ register variables (except maybe for 'env').
		11	+
		12	+- fully convert the x86 target. The minimal amount of work includes:
		13	+ - add cc_src, cc_dst and cc_op as globals
		14	+ - disable its eflags optimization (the liveness analysis should
		15	+ suffice)
		16	+ - move complicated operations to helpers (in particular FPU, SSE, MMX).
		17	+
		18	+- optimize the x86 target:
		19	+ - move some or all the registers as globals
		20	+ - use the TB prologue and epilogue to have QEMU target registers in
		21	+ pre assigned host registers.
		22	+
		23	+Ideas:
		24	+
		25	+- Move the slow part of the qemu_ld/st ops after the end of the TB.
		26	+
		27	+- Experiment: change instruction storage to simplify macro handling
		28	+ and to handle dynamic allocation and see if the translation speed is
		29	+ OK.
		30	+
		31	+- change exception syntax to get closer to QOP system (exception
		32	+ parameters given with a specific instruction).

tcg/i386/tcg-target.c 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+const char *tcg_target_reg_names[TCG_TARGET_NB_REGS] = {
		25	+ "%eax",
		26	+ "%ecx",
		27	+ "%edx",
		28	+ "%ebx",
		29	+ "%esp",
		30	+ "%ebp",
		31	+ "%esi",
		32	+ "%edi",
		33	+};
		34	+
		35	+int tcg_target_reg_alloc_order[TCG_TARGET_NB_REGS] = {
		36	+ TCG_REG_EAX,
		37	+ TCG_REG_EDX,
		38	+ TCG_REG_ECX,
		39	+ TCG_REG_EBX,
		40	+ TCG_REG_ESI,
		41	+ TCG_REG_EDI,
		42	+ TCG_REG_EBP,
		43	+ TCG_REG_ESP,
		44	+};
		45	+
		46	+const int tcg_target_call_iarg_regs[3] = { TCG_REG_EAX, TCG_REG_EDX, TCG_REG_ECX };
		47	+const int tcg_target_call_oarg_regs[2] = { TCG_REG_EAX, TCG_REG_EDX };
		48	+
		49	+static void patch_reloc(uint8_t *code_ptr, int type,
		50	+ tcg_target_long value)
		51	+{
		52	+ switch(type) {
		53	+ case R_386_32:
		54	+ (uint32_t )code_ptr = value;
		55	+ break;
		56	+ case R_386_PC32:
		57	+ (uint32_t )code_ptr = value - (long)code_ptr;
		58	+ break;
		59	+ default:
		60	+ tcg_abort();
		61	+ }
		62	+}
		63	+
		64	+/* maximum number of register used for input function arguments */
		65	+static inline int tcg_target_get_call_iarg_regs_count(int flags)
		66	+{
		67	+ flags &= TCG_CALL_TYPE_MASK;
		68	+ switch(flags) {
		69	+ case TCG_CALL_TYPE_STD:
		70	+ return 0;
		71	+ case TCG_CALL_TYPE_REGPARM_1:
		72	+ case TCG_CALL_TYPE_REGPARM_2:
		73	+ case TCG_CALL_TYPE_REGPARM:
		74	+ return flags - TCG_CALL_TYPE_REGPARM_1 + 1;
		75	+ default:
		76	+ tcg_abort();
		77	+ }
		78	+}
		79	+
		80	+/* parse target specific constraints */
		81	+int target_parse_constraint(TCGArgConstraint ct, const char *pct_str)
		82	+{
		83	+ const char *ct_str;
		84	+
		85	+ ct_str = *pct_str;
		86	+ switch(ct_str[0]) {
		87	+ case 'a':
		88	+ ct->ct \|= TCG_CT_REG;
		89	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_EAX);
		90	+ break;
		91	+ case 'b':
		92	+ ct->ct \|= TCG_CT_REG;
		93	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_EBX);
		94	+ break;
		95	+ case 'c':
		96	+ ct->ct \|= TCG_CT_REG;
		97	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_ECX);
		98	+ break;
		99	+ case 'd':
		100	+ ct->ct \|= TCG_CT_REG;
		101	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_EDX);
		102	+ break;
		103	+ case 'S':
		104	+ ct->ct \|= TCG_CT_REG;
		105	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_ESI);
		106	+ break;
		107	+ case 'D':
		108	+ ct->ct \|= TCG_CT_REG;
		109	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_EDI);
		110	+ break;
		111	+ case 'q':
		112	+ ct->ct \|= TCG_CT_REG;
		113	+ tcg_regset_set32(ct->u.regs, 0, 0xf);
		114	+ break;
		115	+ case 'r':
		116	+ ct->ct \|= TCG_CT_REG;
		117	+ tcg_regset_set32(ct->u.regs, 0, 0xff);
		118	+ break;
		119	+
		120	+ /* qemu_ld/st address constraint */
		121	+ case 'L':
		122	+ ct->ct \|= TCG_CT_REG;
		123	+ tcg_regset_set32(ct->u.regs, 0, 0xff);
		124	+ tcg_regset_reset_reg(ct->u.regs, TCG_REG_EAX);
		125	+ tcg_regset_reset_reg(ct->u.regs, TCG_REG_EDX);
		126	+ break;
		127	+ default:
		128	+ return -1;
		129	+ }
		130	+ ct_str++;
		131	+ *pct_str = ct_str;
		132	+ return 0;
		133	+}
		134	+
		135	+/* test if a constant matches the constraint */
		136	+static inline int tcg_target_const_match(tcg_target_long val,
		137	+ const TCGArgConstraint *arg_ct)
		138	+{
		139	+ int ct;
		140	+ ct = arg_ct->ct;
		141	+ if (ct & TCG_CT_CONST)
		142	+ return 1;
		143	+ else
		144	+ return 0;
		145	+}
		146	+
		147	+#define ARITH_ADD 0
		148	+#define ARITH_OR 1
		149	+#define ARITH_ADC 2
		150	+#define ARITH_SBB 3
		151	+#define ARITH_AND 4
		152	+#define ARITH_SUB 5
		153	+#define ARITH_XOR 6
		154	+#define ARITH_CMP 7
		155	+
		156	+#define SHIFT_SHL 4
		157	+#define SHIFT_SHR 5
		158	+#define SHIFT_SAR 7
		159	+
		160	+#define JCC_JMP (-1)
		161	+#define JCC_JO 0x0
		162	+#define JCC_JNO 0x1
		163	+#define JCC_JB 0x2
		164	+#define JCC_JAE 0x3
		165	+#define JCC_JE 0x4
		166	+#define JCC_JNE 0x5
		167	+#define JCC_JBE 0x6
		168	+#define JCC_JA 0x7
		169	+#define JCC_JS 0x8
		170	+#define JCC_JNS 0x9
		171	+#define JCC_JP 0xa
		172	+#define JCC_JNP 0xb
		173	+#define JCC_JL 0xc
		174	+#define JCC_JGE 0xd
		175	+#define JCC_JLE 0xe
		176	+#define JCC_JG 0xf
		177	+
		178	+#define P_EXT 0x100 /* 0x0f opcode prefix */
		179	+
		180	+static const uint8_t tcg_cond_to_jcc[10] = {
		181	+ [TCG_COND_EQ] = JCC_JE,
		182	+ [TCG_COND_NE] = JCC_JNE,
		183	+ [TCG_COND_LT] = JCC_JL,
		184	+ [TCG_COND_GE] = JCC_JGE,
		185	+ [TCG_COND_LE] = JCC_JLE,
		186	+ [TCG_COND_GT] = JCC_JG,
		187	+ [TCG_COND_LTU] = JCC_JB,
		188	+ [TCG_COND_GEU] = JCC_JAE,
		189	+ [TCG_COND_LEU] = JCC_JBE,
		190	+ [TCG_COND_GTU] = JCC_JA,
		191	+};
		192	+
		193	+static inline void tcg_out_opc(TCGContext *s, int opc)
		194	+{
		195	+ if (opc & P_EXT)
		196	+ tcg_out8(s, 0x0f);
		197	+ tcg_out8(s, opc);
		198	+}
		199	+
		200	+static inline void tcg_out_modrm(TCGContext *s, int opc, int r, int rm)
		201	+{
		202	+ tcg_out_opc(s, opc);
		203	+ tcg_out8(s, 0xc0 \| (r << 3) \| rm);
		204	+}
		205	+
		206	+/* rm == -1 means no register index */
		207	+static inline void tcg_out_modrm_offset(TCGContext *s, int opc, int r, int rm,
		208	+ int32_t offset)
		209	+{
		210	+ tcg_out_opc(s, opc);
		211	+ if (rm == -1) {
		212	+ tcg_out8(s, 0x05 \| (r << 3));
		213	+ tcg_out32(s, offset);
		214	+ } else if (offset == 0 && rm != TCG_REG_EBP) {
		215	+ if (rm == TCG_REG_ESP) {
		216	+ tcg_out8(s, 0x04 \| (r << 3));
		217	+ tcg_out8(s, 0x24);
		218	+ } else {
		219	+ tcg_out8(s, 0x00 \| (r << 3) \| rm);
		220	+ }
		221	+ } else if ((int8_t)offset == offset) {
		222	+ if (rm == TCG_REG_ESP) {
		223	+ tcg_out8(s, 0x44 \| (r << 3));
		224	+ tcg_out8(s, 0x24);
		225	+ } else {
		226	+ tcg_out8(s, 0x40 \| (r << 3) \| rm);
		227	+ }
		228	+ tcg_out8(s, offset);
		229	+ } else {
		230	+ if (rm == TCG_REG_ESP) {
		231	+ tcg_out8(s, 0x84 \| (r << 3));
		232	+ tcg_out8(s, 0x24);
		233	+ } else {
		234	+ tcg_out8(s, 0x80 \| (r << 3) \| rm);
		235	+ }
		236	+ tcg_out32(s, offset);
		237	+ }
		238	+}
		239	+
		240	+static inline void tcg_out_mov(TCGContext *s, int ret, int arg)
		241	+{
		242	+ if (arg != ret)
		243	+ tcg_out_modrm(s, 0x8b, ret, arg);
		244	+}
		245	+
		246	+static inline void tcg_out_movi(TCGContext *s, TCGType type,
		247	+ int ret, int32_t arg)
		248	+{
		249	+ if (arg == 0) {
		250	+ /* xor r0,r0 */
		251	+ tcg_out_modrm(s, 0x01 \| (ARITH_XOR << 3), ret, ret);
		252	+ } else {
		253	+ tcg_out8(s, 0xb8 + ret);
		254	+ tcg_out32(s, arg);
		255	+ }
		256	+}
		257	+
		258	+static inline void tcg_out_ld(TCGContext *s, int ret,
		259	+ int arg1, int32_t arg2)
		260	+{
		261	+ /* movl */
		262	+ tcg_out_modrm_offset(s, 0x8b, ret, arg1, arg2);
		263	+}
		264	+
		265	+static inline void tcg_out_st(TCGContext *s, int arg,
		266	+ int arg1, int32_t arg2)
		267	+{
		268	+ /* movl */
		269	+ tcg_out_modrm_offset(s, 0x89, arg, arg1, arg2);
		270	+}
		271	+
		272	+static inline void tgen_arithi(TCGContext *s, int c, int r0, int32_t val)
		273	+{
		274	+ if (val == (int8_t)val) {
		275	+ tcg_out_modrm(s, 0x83, c, r0);
		276	+ tcg_out8(s, val);
		277	+ } else {
		278	+ tcg_out_modrm(s, 0x81, c, r0);
		279	+ tcg_out32(s, val);
		280	+ }
		281	+}
		282	+
		283	+void tcg_out_addi(TCGContext *s, int reg, tcg_target_long val)
		284	+{
		285	+ if (val != 0)
		286	+ tgen_arithi(s, ARITH_ADD, reg, val);
		287	+}
		288	+
		289	+static void tcg_out_jxx(TCGContext *s, int opc, int label_index)
		290	+{
		291	+ int32_t val, val1;
		292	+ TCGLabel *l = &s->labels[label_index];
		293	+
		294	+ if (l->has_value) {
		295	+ val = l->u.value - (tcg_target_long)s->code_ptr;
		296	+ val1 = val - 2;
		297	+ if ((int8_t)val1 == val1) {
		298	+ if (opc == -1)
		299	+ tcg_out8(s, 0xeb);
		300	+ else
		301	+ tcg_out8(s, 0x70 + opc);
		302	+ tcg_out8(s, val1);
		303	+ } else {
		304	+ if (opc == -1) {
		305	+ tcg_out8(s, 0xe9);
		306	+ tcg_out32(s, val - 5);
		307	+ } else {
		308	+ tcg_out8(s, 0x0f);
		309	+ tcg_out8(s, 0x80 + opc);
		310	+ tcg_out32(s, val - 6);
		311	+ }
		312	+ }
		313	+ } else {
		314	+ if (opc == -1) {
		315	+ tcg_out8(s, 0xe9);
		316	+ } else {
		317	+ tcg_out8(s, 0x0f);
		318	+ tcg_out8(s, 0x80 + opc);
		319	+ }
		320	+ tcg_out_reloc(s, s->code_ptr, R_386_PC32, label_index, -4);
		321	+ tcg_out32(s, -4);
		322	+ }
		323	+}
		324	+
		325	+static void tcg_out_brcond(TCGContext *s, int cond,
		326	+ TCGArg arg1, TCGArg arg2, int const_arg2,
		327	+ int label_index)
		328	+{
		329	+ int c;
		330	+ if (const_arg2) {
		331	+ if (arg2 == 0) {
		332	+ /* use test */
		333	+ switch(cond) {
		334	+ case TCG_COND_EQ:
		335	+ c = JCC_JNE;
		336	+ break;
		337	+ case TCG_COND_NE:
		338	+ c = JCC_JNE;
		339	+ break;
		340	+ case TCG_COND_LT:
		341	+ c = JCC_JS;
		342	+ break;
		343	+ case TCG_COND_GE:
		344	+ c = JCC_JNS;
		345	+ break;
		346	+ default:
		347	+ goto do_cmpi;
		348	+ }
		349	+ /* test r, r */
		350	+ tcg_out_modrm(s, 0x85, arg1, arg1);
		351	+ tcg_out_jxx(s, c, label_index);
		352	+ } else {
		353	+ do_cmpi:
		354	+ tgen_arithi(s, ARITH_CMP, arg1, arg2);
		355	+ tcg_out_jxx(s, tcg_cond_to_jcc[cond], label_index);
		356	+ }
		357	+ } else {
		358	+ tcg_out_modrm(s, 0x01 \| (ARITH_CMP << 3), arg1, arg2);
		359	+ tcg_out_jxx(s, tcg_cond_to_jcc[cond], label_index);
		360	+ }
		361	+}
		362	+
		363	+/* XXX: we implement it at the target level to avoid having to
		364	+ handle cross basic blocks temporaries */
		365	+static void tcg_out_brcond2(TCGContext *s,
		366	+ const TCGArg args, const int const_args)
		367	+{
		368	+ int label_next;
		369	+ label_next = gen_new_label();
		370	+ switch(args[4]) {
		371	+ case TCG_COND_EQ:
		372	+ tcg_out_brcond(s, TCG_COND_NE, args[0], args[2], const_args[2], label_next);
		373	+ tcg_out_brcond(s, TCG_COND_EQ, args[1], args[3], const_args[3], args[5]);
		374	+ break;
		375	+ case TCG_COND_NE:
		376	+ tcg_out_brcond(s, TCG_COND_NE, args[0], args[2], const_args[2], args[5]);
		377	+ tcg_out_brcond(s, TCG_COND_EQ, args[1], args[3], const_args[3], label_next);
		378	+ break;
		379	+ case TCG_COND_LT:
		380	+ tcg_out_brcond(s, TCG_COND_LT, args[1], args[3], const_args[3], args[5]);
		381	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		382	+ tcg_out_brcond(s, TCG_COND_LT, args[0], args[2], const_args[2], args[5]);
		383	+ break;
		384	+ case TCG_COND_LE:
		385	+ tcg_out_brcond(s, TCG_COND_LT, args[1], args[3], const_args[3], args[5]);
		386	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		387	+ tcg_out_brcond(s, TCG_COND_LE, args[0], args[2], const_args[2], args[5]);
		388	+ break;
		389	+ case TCG_COND_GT:
		390	+ tcg_out_brcond(s, TCG_COND_GT, args[1], args[3], const_args[3], args[5]);
		391	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		392	+ tcg_out_brcond(s, TCG_COND_GT, args[0], args[2], const_args[2], args[5]);
		393	+ break;
		394	+ case TCG_COND_GE:
		395	+ tcg_out_brcond(s, TCG_COND_GT, args[1], args[3], const_args[3], args[5]);
		396	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		397	+ tcg_out_brcond(s, TCG_COND_GE, args[0], args[2], const_args[2], args[5]);
		398	+ break;
		399	+ case TCG_COND_LTU:
		400	+ tcg_out_brcond(s, TCG_COND_LTU, args[1], args[3], const_args[3], args[5]);
		401	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		402	+ tcg_out_brcond(s, TCG_COND_LTU, args[0], args[2], const_args[2], args[5]);
		403	+ break;
		404	+ case TCG_COND_LEU:
		405	+ tcg_out_brcond(s, TCG_COND_LTU, args[1], args[3], const_args[3], args[5]);
		406	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		407	+ tcg_out_brcond(s, TCG_COND_LEU, args[0], args[2], const_args[2], args[5]);
		408	+ break;
		409	+ case TCG_COND_GTU:
		410	+ tcg_out_brcond(s, TCG_COND_GTU, args[1], args[3], const_args[3], args[5]);
		411	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		412	+ tcg_out_brcond(s, TCG_COND_GTU, args[0], args[2], const_args[2], args[5]);
		413	+ break;
		414	+ case TCG_COND_GEU:
		415	+ tcg_out_brcond(s, TCG_COND_GTU, args[1], args[3], const_args[3], args[5]);
		416	+ tcg_out_brcond(s, TCG_COND_NE, args[1], args[3], const_args[3], label_next);
		417	+ tcg_out_brcond(s, TCG_COND_GEU, args[0], args[2], const_args[2], args[5]);
		418	+ break;
		419	+ default:
		420	+ tcg_abort();
		421	+ }
		422	+ tcg_out_label(s, label_next, (tcg_target_long)s->code_ptr);
		423	+}
		424	+
		425	+#if defined(CONFIG_SOFTMMU)
		426	+extern void __ldb_mmu(void);
		427	+extern void __ldw_mmu(void);
		428	+extern void __ldl_mmu(void);
		429	+extern void __ldq_mmu(void);
		430	+
		431	+extern void __stb_mmu(void);
		432	+extern void __stw_mmu(void);
		433	+extern void __stl_mmu(void);
		434	+extern void __stq_mmu(void);
		435	+
		436	+static void *qemu_ld_helpers[4] = {
		437	+ __ldb_mmu,
		438	+ __ldw_mmu,
		439	+ __ldl_mmu,
		440	+ __ldq_mmu,
		441	+};
		442	+
		443	+static void *qemu_st_helpers[4] = {
		444	+ __stb_mmu,
		445	+ __stw_mmu,
		446	+ __stl_mmu,
		447	+ __stq_mmu,
		448	+};
		449	+#endif
		450	+
		451	+/* XXX: qemu_ld and qemu_st could be modified to clobber only EDX and
		452	+ EAX. It will be useful once fixed registers globals are less
		453	+ common. */
		454	+static void tcg_out_qemu_ld(TCGContext s, const TCGArg args,
		455	+ int opc)
		456	+{
		457	+ int addr_reg, data_reg, data_reg2, r0, r1, mem_index, s_bits, bswap;
		458	+#if defined(CONFIG_SOFTMMU)
		459	+ uint8_t label1_ptr, label2_ptr;
		460	+#endif
		461	+#if TARGET_LONG_BITS == 64
		462	+#if defined(CONFIG_SOFTMMU)
		463	+ uint8_t *label3_ptr;
		464	+#endif
		465	+ int addr_reg2;
		466	+#endif
		467	+
		468	+ data_reg = *args++;
		469	+ if (opc == 3)
		470	+ data_reg2 = *args++;
		471	+ else
		472	+ data_reg2 = 0;
		473	+ addr_reg = *args++;
		474	+#if TARGET_LONG_BITS == 64
		475	+ addr_reg2 = *args++;
		476	+#endif
		477	+ mem_index = *args;
		478	+ s_bits = opc & 3;
		479	+
		480	+ r0 = TCG_REG_EAX;
		481	+ r1 = TCG_REG_EDX;
		482	+
		483	+#if defined(CONFIG_SOFTMMU)
		484	+ tcg_out_mov(s, r1, addr_reg);
		485	+
		486	+ tcg_out_mov(s, r0, addr_reg);
		487	+
		488	+ tcg_out_modrm(s, 0xc1, 5, r1); /* shr $x, r1 */
		489	+ tcg_out8(s, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS);
		490	+
		491	+ tcg_out_modrm(s, 0x81, 4, r0); /* andl $x, r0 */
		492	+ tcg_out32(s, TARGET_PAGE_MASK \| ((1 << s_bits) - 1));
		493	+
		494	+ tcg_out_modrm(s, 0x81, 4, r1); /* andl $x, r1 */
		495	+ tcg_out32(s, (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS);
		496	+
		497	+ tcg_out_opc(s, 0x8d); /* lea offset(r1, %ebp), r1 */
		498	+ tcg_out8(s, 0x80 \| (r1 << 3) \| 0x04);
		499	+ tcg_out8(s, (5 << 3) \| r1);
		500	+ tcg_out32(s, offsetof(CPUState, tlb_table[mem_index][0].addr_read));
		501	+
		502	+ /* cmp 0(r1), r0 */
		503	+ tcg_out_modrm_offset(s, 0x3b, r0, r1, 0);
		504	+
		505	+ tcg_out_mov(s, r0, addr_reg);
		506	+
		507	+#if TARGET_LONG_BITS == 32
		508	+ /* je label1 */
		509	+ tcg_out8(s, 0x70 + JCC_JE);
		510	+ label1_ptr = s->code_ptr;
		511	+ s->code_ptr++;
		512	+#else
		513	+ /* jne label3 */
		514	+ tcg_out8(s, 0x70 + JCC_JNE);
		515	+ label3_ptr = s->code_ptr;
		516	+ s->code_ptr++;
		517	+
		518	+ /* cmp 4(r1), addr_reg2 */
		519	+ tcg_out_modrm_offset(s, 0x3b, addr_reg2, r1, 4);
		520	+
		521	+ /* je label1 */
		522	+ tcg_out8(s, 0x70 + JCC_JE);
		523	+ label1_ptr = s->code_ptr;
		524	+ s->code_ptr++;
		525	+
		526	+ /* label3: */
		527	+ *label3_ptr = s->code_ptr - label3_ptr - 1;
		528	+#endif
		529	+
		530	+ /* XXX: move that code at the end of the TB */
		531	+#if TARGET_LONG_BITS == 32
		532	+ tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_EDX, mem_index);
		533	+#else
		534	+ tcg_out_mov(s, TCG_REG_EDX, addr_reg2);
		535	+ tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_ECX, mem_index);
		536	+#endif
		537	+ tcg_out8(s, 0xe8);
		538	+ tcg_out32(s, (tcg_target_long)qemu_ld_helpers[s_bits] -
		539	+ (tcg_target_long)s->code_ptr - 4);
		540	+
		541	+ switch(opc) {
		542	+ case 0 \| 4:
		543	+ /* movsbl */
		544	+ tcg_out_modrm(s, 0xbe \| P_EXT, data_reg, TCG_REG_EAX);
		545	+ break;
		546	+ case 1 \| 4:
		547	+ /* movswl */
		548	+ tcg_out_modrm(s, 0xbf \| P_EXT, data_reg, TCG_REG_EAX);
		549	+ break;
		550	+ case 0:
		551	+ case 1:
		552	+ case 2:
		553	+ default:
		554	+ tcg_out_mov(s, data_reg, TCG_REG_EAX);
		555	+ break;
		556	+ case 3:
		557	+ if (data_reg == TCG_REG_EDX) {
		558	+ tcg_out_opc(s, 0x90 + TCG_REG_EDX); /* xchg %edx, %eax */
		559	+ tcg_out_mov(s, data_reg2, TCG_REG_EAX);
		560	+ } else {
		561	+ tcg_out_mov(s, data_reg, TCG_REG_EAX);
		562	+ tcg_out_mov(s, data_reg2, TCG_REG_EDX);
		563	+ }
		564	+ break;
		565	+ }
		566	+
		567	+ /* jmp label2 */
		568	+ tcg_out8(s, 0xeb);
		569	+ label2_ptr = s->code_ptr;
		570	+ s->code_ptr++;
		571	+
		572	+ /* label1: */
		573	+ *label1_ptr = s->code_ptr - label1_ptr - 1;
		574	+
		575	+ /* add x(r1), r0 */
		576	+ tcg_out_modrm_offset(s, 0x03, r0, r1, offsetof(CPUTLBEntry, addend) -
		577	+ offsetof(CPUTLBEntry, addr_read));
		578	+#else
		579	+ r0 = addr_reg;
		580	+#endif
		581	+
		582	+#ifdef TARGET_WORDS_BIGENDIAN
		583	+ bswap = 1;
		584	+#else
		585	+ bswap = 0;
		586	+#endif
		587	+ switch(opc) {
		588	+ case 0:
		589	+ /* movzbl */
		590	+ tcg_out_modrm_offset(s, 0xb6 \| P_EXT, data_reg, r0, 0);
		591	+ break;
		592	+ case 0 \| 4:
		593	+ /* movsbl */
		594	+ tcg_out_modrm_offset(s, 0xbe \| P_EXT, data_reg, r0, 0);
		595	+ break;
		596	+ case 1:
		597	+ /* movzwl */
		598	+ tcg_out_modrm_offset(s, 0xb7 \| P_EXT, data_reg, r0, 0);
		599	+ if (bswap) {
		600	+ /* rolw $8, data_reg */
		601	+ tcg_out8(s, 0x66);
		602	+ tcg_out_modrm(s, 0xc1, 0, data_reg);
		603	+ tcg_out8(s, 8);
		604	+ }
		605	+ break;
		606	+ case 1 \| 4:
		607	+ /* movswl */
		608	+ tcg_out_modrm_offset(s, 0xbf \| P_EXT, data_reg, r0, 0);
		609	+ if (bswap) {
		610	+ /* rolw $8, data_reg */
		611	+ tcg_out8(s, 0x66);
		612	+ tcg_out_modrm(s, 0xc1, 0, data_reg);
		613	+ tcg_out8(s, 8);
		614	+
		615	+ /* movswl data_reg, data_reg */
		616	+ tcg_out_modrm(s, 0xbf \| P_EXT, data_reg, data_reg);
		617	+ }
		618	+ break;
		619	+ case 2:
		620	+ /* movl (r0), data_reg */
		621	+ tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 0);
		622	+ if (bswap) {
		623	+ /* bswap */
		624	+ tcg_out_opc(s, (0xc8 + data_reg) \| P_EXT);
		625	+ }
		626	+ break;
		627	+ case 3:
		628	+ /* XXX: could be nicer */
		629	+ if (r0 == data_reg) {
		630	+ r1 = TCG_REG_EDX;
		631	+ if (r1 == data_reg)
		632	+ r1 = TCG_REG_EAX;
		633	+ tcg_out_mov(s, r1, r0);
		634	+ r0 = r1;
		635	+ }
		636	+ if (!bswap) {
		637	+ tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 0);
		638	+ tcg_out_modrm_offset(s, 0x8b, data_reg2, r0, 4);
		639	+ } else {
		640	+ tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 4);
		641	+ tcg_out_opc(s, (0xc8 + data_reg) \| P_EXT);
		642	+
		643	+ tcg_out_modrm_offset(s, 0x8b, data_reg2, r0, 0);
		644	+ /* bswap */
		645	+ tcg_out_opc(s, (0xc8 + data_reg2) \| P_EXT);
		646	+ }
		647	+ break;
		648	+ default:
		649	+ tcg_abort();
		650	+ }
		651	+
		652	+#if defined(CONFIG_SOFTMMU)
		653	+ /* label2: */
		654	+ *label2_ptr = s->code_ptr - label2_ptr - 1;
		655	+#endif
		656	+}
		657	+
		658	+
		659	+static void tcg_out_qemu_st(TCGContext s, const TCGArg args,
		660	+ int opc)
		661	+{
		662	+ int addr_reg, data_reg, data_reg2, r0, r1, mem_index, s_bits, bswap;
		663	+#if defined(CONFIG_SOFTMMU)
		664	+ uint8_t label1_ptr, label2_ptr;
		665	+#endif
		666	+#if TARGET_LONG_BITS == 64
		667	+#if defined(CONFIG_SOFTMMU)
		668	+ uint8_t *label3_ptr;
		669	+#endif
		670	+ int addr_reg2;
		671	+#endif
		672	+
		673	+ data_reg = *args++;
		674	+ if (opc == 3)
		675	+ data_reg2 = *args++;
		676	+ else
		677	+ data_reg2 = 0;
		678	+ addr_reg = *args++;
		679	+#if TARGET_LONG_BITS == 64
		680	+ addr_reg2 = *args++;
		681	+#endif
		682	+ mem_index = *args;
		683	+
		684	+ s_bits = opc;
		685	+
		686	+ r0 = TCG_REG_EAX;
		687	+ r1 = TCG_REG_EDX;
		688	+
		689	+#if defined(CONFIG_SOFTMMU)
		690	+ tcg_out_mov(s, r1, addr_reg);
		691	+
		692	+ tcg_out_mov(s, r0, addr_reg);
		693	+
		694	+ tcg_out_modrm(s, 0xc1, 5, r1); /* shr $x, r1 */
		695	+ tcg_out8(s, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS);
		696	+
		697	+ tcg_out_modrm(s, 0x81, 4, r0); /* andl $x, r0 */
		698	+ tcg_out32(s, TARGET_PAGE_MASK \| ((1 << s_bits) - 1));
		699	+
		700	+ tcg_out_modrm(s, 0x81, 4, r1); /* andl $x, r1 */
		701	+ tcg_out32(s, (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS);
		702	+
		703	+ tcg_out_opc(s, 0x8d); /* lea offset(r1, %ebp), r1 */
		704	+ tcg_out8(s, 0x80 \| (r1 << 3) \| 0x04);
		705	+ tcg_out8(s, (5 << 3) \| r1);
		706	+ tcg_out32(s, offsetof(CPUState, tlb_table[mem_index][0].addr_write));
		707	+
		708	+ /* cmp 0(r1), r0 */
		709	+ tcg_out_modrm_offset(s, 0x3b, r0, r1, 0);
		710	+
		711	+ tcg_out_mov(s, r0, addr_reg);
		712	+
		713	+#if TARGET_LONG_BITS == 32
		714	+ /* je label1 */
		715	+ tcg_out8(s, 0x70 + JCC_JE);
		716	+ label1_ptr = s->code_ptr;
		717	+ s->code_ptr++;
		718	+#else
		719	+ /* jne label3 */
		720	+ tcg_out8(s, 0x70 + JCC_JNE);
		721	+ label3_ptr = s->code_ptr;
		722	+ s->code_ptr++;
		723	+
		724	+ /* cmp 4(r1), addr_reg2 */
		725	+ tcg_out_modrm_offset(s, 0x3b, addr_reg2, r1, 4);
		726	+
		727	+ /* je label1 */
		728	+ tcg_out8(s, 0x70 + JCC_JE);
		729	+ label1_ptr = s->code_ptr;
		730	+ s->code_ptr++;
		731	+
		732	+ /* label3: */
		733	+ *label3_ptr = s->code_ptr - label3_ptr - 1;
		734	+#endif
		735	+
		736	+ /* XXX: move that code at the end of the TB */
		737	+#if TARGET_LONG_BITS == 32
		738	+ if (opc == 3) {
		739	+ tcg_out_mov(s, TCG_REG_EDX, data_reg);
		740	+ tcg_out_mov(s, TCG_REG_ECX, data_reg2);
		741	+ tcg_out8(s, 0x6a); /* push Ib */
		742	+ tcg_out8(s, mem_index);
		743	+ tcg_out8(s, 0xe8);
		744	+ tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] -
		745	+ (tcg_target_long)s->code_ptr - 4);
		746	+ tcg_out_addi(s, TCG_REG_ESP, 4);
		747	+ } else {
		748	+ switch(opc) {
		749	+ case 0:
		750	+ /* movzbl */
		751	+ tcg_out_modrm(s, 0xb6 \| P_EXT, TCG_REG_EDX, data_reg);
		752	+ break;
		753	+ case 1:
		754	+ /* movzwl */
		755	+ tcg_out_modrm(s, 0xb7 \| P_EXT, TCG_REG_EDX, data_reg);
		756	+ break;
		757	+ case 2:
		758	+ tcg_out_mov(s, TCG_REG_EDX, data_reg);
		759	+ break;
		760	+ }
		761	+ tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_ECX, mem_index);
		762	+ tcg_out8(s, 0xe8);
		763	+ tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] -
		764	+ (tcg_target_long)s->code_ptr - 4);
		765	+ }
		766	+#else
		767	+ if (opc == 3) {
		768	+ tcg_out_mov(s, TCG_REG_EDX, addr_reg2);
		769	+ tcg_out8(s, 0x6a); /* push Ib */
		770	+ tcg_out8(s, mem_index);
		771	+ tcg_out_opc(s, 0x50 + data_reg2); /* push */
		772	+ tcg_out_opc(s, 0x50 + data_reg); /* push */
		773	+ tcg_out8(s, 0xe8);
		774	+ tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] -
		775	+ (tcg_target_long)s->code_ptr - 4);
		776	+ tcg_out_addi(s, TCG_REG_ESP, 12);
		777	+ } else {
		778	+ tcg_out_mov(s, TCG_REG_EDX, addr_reg2);
		779	+ switch(opc) {
		780	+ case 0:
		781	+ /* movzbl */
		782	+ tcg_out_modrm(s, 0xb6 \| P_EXT, TCG_REG_ECX, data_reg);
		783	+ break;
		784	+ case 1:
		785	+ /* movzwl */
		786	+ tcg_out_modrm(s, 0xb7 \| P_EXT, TCG_REG_ECX, data_reg);
		787	+ break;
		788	+ case 2:
		789	+ tcg_out_mov(s, TCG_REG_ECX, data_reg);
		790	+ break;
		791	+ }
		792	+ tcg_out8(s, 0x6a); /* push Ib */
		793	+ tcg_out8(s, mem_index);
		794	+ tcg_out8(s, 0xe8);
		795	+ tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] -
		796	+ (tcg_target_long)s->code_ptr - 4);
		797	+ tcg_out_addi(s, TCG_REG_ESP, 4);
		798	+ }
		799	+#endif
		800	+
		801	+ /* jmp label2 */
		802	+ tcg_out8(s, 0xeb);
		803	+ label2_ptr = s->code_ptr;
		804	+ s->code_ptr++;
		805	+
		806	+ /* label1: */
		807	+ *label1_ptr = s->code_ptr - label1_ptr - 1;
		808	+
		809	+ /* add x(r1), r0 */
		810	+ tcg_out_modrm_offset(s, 0x03, r0, r1, offsetof(CPUTLBEntry, addend) -
		811	+ offsetof(CPUTLBEntry, addr_write));
		812	+#else
		813	+ r0 = addr_reg;
		814	+#endif
		815	+
		816	+#ifdef TARGET_WORDS_BIGENDIAN
		817	+ bswap = 1;
		818	+#else
		819	+ bswap = 0;
		820	+#endif
		821	+ switch(opc) {
		822	+ case 0:
		823	+ /* movb */
		824	+ tcg_out_modrm_offset(s, 0x88, data_reg, r0, 0);
		825	+ break;
		826	+ case 1:
		827	+ if (bswap) {
		828	+ tcg_out_mov(s, r1, data_reg);
		829	+ tcg_out8(s, 0x66); /* rolw $8, %ecx */
		830	+ tcg_out_modrm(s, 0xc1, 0, r1);
		831	+ tcg_out8(s, 8);
		832	+ data_reg = r1;
		833	+ }
		834	+ /* movw */
		835	+ tcg_out8(s, 0x66);
		836	+ tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);
		837	+ break;
		838	+ case 2:
		839	+ if (bswap) {
		840	+ tcg_out_mov(s, r1, data_reg);
		841	+ /* bswap data_reg */
		842	+ tcg_out_opc(s, (0xc8 + r1) \| P_EXT);
		843	+ data_reg = r1;
		844	+ }
		845	+ /* movl */
		846	+ tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);
		847	+ break;
		848	+ case 3:
		849	+ if (bswap) {
		850	+ tcg_out_mov(s, r1, data_reg2);
		851	+ /* bswap data_reg */
		852	+ tcg_out_opc(s, (0xc8 + r1) \| P_EXT);
		853	+ tcg_out_modrm_offset(s, 0x89, r1, r0, 0);
		854	+ tcg_out_mov(s, r1, data_reg);
		855	+ /* bswap data_reg */
		856	+ tcg_out_opc(s, (0xc8 + r1) \| P_EXT);
		857	+ tcg_out_modrm_offset(s, 0x89, r1, r0, 4);
		858	+ } else {
		859	+ tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);
		860	+ tcg_out_modrm_offset(s, 0x89, data_reg2, r0, 4);
		861	+ }
		862	+ break;
		863	+ default:
		864	+ tcg_abort();
		865	+ }
		866	+
		867	+#if defined(CONFIG_SOFTMMU)
		868	+ /* label2: */
		869	+ *label2_ptr = s->code_ptr - label2_ptr - 1;
		870	+#endif
		871	+}
		872	+
		873	+static inline void tcg_out_op(TCGContext *s, int opc,
		874	+ const TCGArg args, const int const_args)
		875	+{
		876	+ int c;
		877	+
		878	+ switch(opc) {
		879	+ case INDEX_op_exit_tb:
		880	+ tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_EAX, args[0]);
		881	+ tcg_out8(s, 0xc3); /* ret */
		882	+ break;
		883	+ case INDEX_op_goto_tb:
		884	+ if (s->tb_jmp_offset) {
		885	+ /* direct jump method */
		886	+ tcg_out8(s, 0xe9); /* jmp im */
		887	+ s->tb_jmp_offset[args[0]] = s->code_ptr - s->code_buf;
		888	+ tcg_out32(s, 0);
		889	+ } else {
		890	+ /* indirect jump method */
		891	+ /* jmp Ev */
		892	+ tcg_out_modrm_offset(s, 0xff, 4, -1,
		893	+ (tcg_target_long)(s->tb_next + args[0]));
		894	+ }
		895	+ s->tb_next_offset[args[0]] = s->code_ptr - s->code_buf;
		896	+ break;
		897	+ case INDEX_op_call:
		898	+ if (const_args[0]) {
		899	+ tcg_out8(s, 0xe8);
		900	+ tcg_out32(s, args[0] - (tcg_target_long)s->code_ptr - 4);
		901	+ } else {
		902	+ tcg_out_modrm(s, 0xff, 2, args[0]);
		903	+ }
		904	+ break;
		905	+ case INDEX_op_jmp:
		906	+ if (const_args[0]) {
		907	+ tcg_out8(s, 0xe9);
		908	+ tcg_out32(s, args[0] - (tcg_target_long)s->code_ptr - 4);
		909	+ } else {
		910	+ tcg_out_modrm(s, 0xff, 4, args[0]);
		911	+ }
		912	+ break;
		913	+ case INDEX_op_br:
		914	+ tcg_out_jxx(s, JCC_JMP, args[0]);
		915	+ break;
		916	+ case INDEX_op_movi_i32:
		917	+ tcg_out_movi(s, TCG_TYPE_I32, args[0], args[1]);
		918	+ break;
		919	+ case INDEX_op_ld8u_i32:
		920	+ /* movzbl */
		921	+ tcg_out_modrm_offset(s, 0xb6 \| P_EXT, args[0], args[1], args[2]);
		922	+ break;
		923	+ case INDEX_op_ld8s_i32:
		924	+ /* movsbl */
		925	+ tcg_out_modrm_offset(s, 0xbe \| P_EXT, args[0], args[1], args[2]);
		926	+ break;
		927	+ case INDEX_op_ld16u_i32:
		928	+ /* movzwl */
		929	+ tcg_out_modrm_offset(s, 0xb7 \| P_EXT, args[0], args[1], args[2]);
		930	+ break;
		931	+ case INDEX_op_ld16s_i32:
		932	+ /* movswl */
		933	+ tcg_out_modrm_offset(s, 0xbf \| P_EXT, args[0], args[1], args[2]);
		934	+ break;
		935	+ case INDEX_op_ld_i32:
		936	+ /* movl */
		937	+ tcg_out_modrm_offset(s, 0x8b, args[0], args[1], args[2]);
		938	+ break;
		939	+ case INDEX_op_st8_i32:
		940	+ /* movb */
		941	+ tcg_out_modrm_offset(s, 0x88, args[0], args[1], args[2]);
		942	+ break;
		943	+ case INDEX_op_st16_i32:
		944	+ /* movw */
		945	+ tcg_out8(s, 0x66);
		946	+ tcg_out_modrm_offset(s, 0x89, args[0], args[1], args[2]);
		947	+ break;
		948	+ case INDEX_op_st_i32:
		949	+ /* movl */
		950	+ tcg_out_modrm_offset(s, 0x89, args[0], args[1], args[2]);
		951	+ break;
		952	+ case INDEX_op_sub_i32:
		953	+ c = ARITH_SUB;
		954	+ goto gen_arith;
		955	+ case INDEX_op_and_i32:
		956	+ c = ARITH_AND;
		957	+ goto gen_arith;
		958	+ case INDEX_op_or_i32:
		959	+ c = ARITH_OR;
		960	+ goto gen_arith;
		961	+ case INDEX_op_xor_i32:
		962	+ c = ARITH_XOR;
		963	+ goto gen_arith;
		964	+ case INDEX_op_add_i32:
		965	+ c = ARITH_ADD;
		966	+ gen_arith:
		967	+ if (const_args[2]) {
		968	+ tgen_arithi(s, c, args[0], args[2]);
		969	+ } else {
		970	+ tcg_out_modrm(s, 0x01 \| (c << 3), args[2], args[0]);
		971	+ }
		972	+ break;
		973	+ case INDEX_op_mul_i32:
		974	+ if (const_args[2]) {
		975	+ int32_t val;
		976	+ val = args[2];
		977	+ if (val == (int8_t)val) {
		978	+ tcg_out_modrm(s, 0x6b, args[0], args[0]);
		979	+ tcg_out8(s, val);
		980	+ } else {
		981	+ tcg_out_modrm(s, 0x69, args[0], args[0]);
		982	+ tcg_out32(s, val);
		983	+ }
		984	+ } else {
		985	+ tcg_out_modrm(s, 0xaf \| P_EXT, args[0], args[2]);
		986	+ }
		987	+ break;
		988	+ case INDEX_op_mulu2_i32:
		989	+ tcg_out_modrm(s, 0xf7, 4, args[3]);
		990	+ break;
		991	+ case INDEX_op_div2_i32:
		992	+ tcg_out_modrm(s, 0xf7, 7, args[4]);
		993	+ break;
		994	+ case INDEX_op_divu2_i32:
		995	+ tcg_out_modrm(s, 0xf7, 6, args[4]);
		996	+ break;
		997	+ case INDEX_op_shl_i32:
		998	+ c = SHIFT_SHL;
		999	+ gen_shift32:
		1000	+ if (const_args[2]) {
		1001	+ if (args[2] == 1) {
		1002	+ tcg_out_modrm(s, 0xd1, c, args[0]);
		1003	+ } else {
		1004	+ tcg_out_modrm(s, 0xc1, c, args[0]);
		1005	+ tcg_out8(s, args[2]);
		1006	+ }
		1007	+ } else {
		1008	+ tcg_out_modrm(s, 0xd3, c, args[0]);
		1009	+ }
		1010	+ break;
		1011	+ case INDEX_op_shr_i32:
		1012	+ c = SHIFT_SHR;
		1013	+ goto gen_shift32;
		1014	+ case INDEX_op_sar_i32:
		1015	+ c = SHIFT_SAR;
		1016	+ goto gen_shift32;
		1017	+
		1018	+ case INDEX_op_add2_i32:
		1019	+ if (const_args[4])
		1020	+ tgen_arithi(s, ARITH_ADD, args[0], args[4]);
		1021	+ else
		1022	+ tcg_out_modrm(s, 0x01 \| (ARITH_ADD << 3), args[4], args[0]);
		1023	+ if (const_args[5])
		1024	+ tgen_arithi(s, ARITH_ADC, args[1], args[5]);
		1025	+ else
		1026	+ tcg_out_modrm(s, 0x01 \| (ARITH_ADC << 3), args[5], args[1]);
		1027	+ break;
		1028	+ case INDEX_op_sub2_i32:
		1029	+ if (const_args[4])
		1030	+ tgen_arithi(s, ARITH_SUB, args[0], args[4]);
		1031	+ else
		1032	+ tcg_out_modrm(s, 0x01 \| (ARITH_SUB << 3), args[4], args[0]);
		1033	+ if (const_args[5])
		1034	+ tgen_arithi(s, ARITH_SBB, args[1], args[5]);
		1035	+ else
		1036	+ tcg_out_modrm(s, 0x01 \| (ARITH_SBB << 3), args[5], args[1]);
		1037	+ break;
		1038	+ case INDEX_op_brcond_i32:
		1039	+ tcg_out_brcond(s, args[2], args[0], args[1], const_args[1], args[3]);
		1040	+ break;
		1041	+ case INDEX_op_brcond2_i32:
		1042	+ tcg_out_brcond2(s, args, const_args);
		1043	+ break;
		1044	+
		1045	+ case INDEX_op_qemu_ld8u:
		1046	+ tcg_out_qemu_ld(s, args, 0);
		1047	+ break;
		1048	+ case INDEX_op_qemu_ld8s:
		1049	+ tcg_out_qemu_ld(s, args, 0 \| 4);
		1050	+ break;
		1051	+ case INDEX_op_qemu_ld16u:
		1052	+ tcg_out_qemu_ld(s, args, 1);
		1053	+ break;
		1054	+ case INDEX_op_qemu_ld16s:
		1055	+ tcg_out_qemu_ld(s, args, 1 \| 4);
		1056	+ break;
		1057	+ case INDEX_op_qemu_ld32u:
		1058	+ tcg_out_qemu_ld(s, args, 2);
		1059	+ break;
		1060	+ case INDEX_op_qemu_ld64:
		1061	+ tcg_out_qemu_ld(s, args, 3);
		1062	+ break;
		1063	+
		1064	+ case INDEX_op_qemu_st8:
		1065	+ tcg_out_qemu_st(s, args, 0);
		1066	+ break;
		1067	+ case INDEX_op_qemu_st16:
		1068	+ tcg_out_qemu_st(s, args, 1);
		1069	+ break;
		1070	+ case INDEX_op_qemu_st32:
		1071	+ tcg_out_qemu_st(s, args, 2);
		1072	+ break;
		1073	+ case INDEX_op_qemu_st64:
		1074	+ tcg_out_qemu_st(s, args, 3);
		1075	+ break;
		1076	+
		1077	+ default:
		1078	+ tcg_abort();
		1079	+ }
		1080	+}
		1081	+
		1082	+static const TCGTargetOpDef x86_op_defs[] = {
		1083	+ { INDEX_op_exit_tb, { } },
		1084	+ { INDEX_op_goto_tb, { } },
		1085	+ { INDEX_op_call, { "ri" } },
		1086	+ { INDEX_op_jmp, { "ri" } },
		1087	+ { INDEX_op_br, { } },
		1088	+ { INDEX_op_mov_i32, { "r", "r" } },
		1089	+ { INDEX_op_movi_i32, { "r" } },
		1090	+ { INDEX_op_ld8u_i32, { "r", "r" } },
		1091	+ { INDEX_op_ld8s_i32, { "r", "r" } },
		1092	+ { INDEX_op_ld16u_i32, { "r", "r" } },
		1093	+ { INDEX_op_ld16s_i32, { "r", "r" } },
		1094	+ { INDEX_op_ld_i32, { "r", "r" } },
		1095	+ { INDEX_op_st8_i32, { "q", "r" } },
		1096	+ { INDEX_op_st16_i32, { "r", "r" } },
		1097	+ { INDEX_op_st_i32, { "r", "r" } },
		1098	+
		1099	+ { INDEX_op_add_i32, { "r", "0", "ri" } },
		1100	+ { INDEX_op_sub_i32, { "r", "0", "ri" } },
		1101	+ { INDEX_op_mul_i32, { "r", "0", "ri" } },
		1102	+ { INDEX_op_mulu2_i32, { "a", "d", "a", "r" } },
		1103	+ { INDEX_op_div2_i32, { "a", "d", "0", "1", "r" } },
		1104	+ { INDEX_op_divu2_i32, { "a", "d", "0", "1", "r" } },
		1105	+ { INDEX_op_and_i32, { "r", "0", "ri" } },
		1106	+ { INDEX_op_or_i32, { "r", "0", "ri" } },
		1107	+ { INDEX_op_xor_i32, { "r", "0", "ri" } },
		1108	+
		1109	+ { INDEX_op_shl_i32, { "r", "0", "ci" } },
		1110	+ { INDEX_op_shr_i32, { "r", "0", "ci" } },
		1111	+ { INDEX_op_sar_i32, { "r", "0", "ci" } },
		1112	+
		1113	+ { INDEX_op_brcond_i32, { "r", "ri" } },
		1114	+
		1115	+ { INDEX_op_add2_i32, { "r", "r", "0", "1", "ri", "ri" } },
		1116	+ { INDEX_op_sub2_i32, { "r", "r", "0", "1", "ri", "ri" } },
		1117	+ { INDEX_op_brcond2_i32, { "r", "r", "ri", "ri" } },
		1118	+
		1119	+#if TARGET_LONG_BITS == 32
		1120	+ { INDEX_op_qemu_ld8u, { "r", "L" } },
		1121	+ { INDEX_op_qemu_ld8s, { "r", "L" } },
		1122	+ { INDEX_op_qemu_ld16u, { "r", "L" } },
		1123	+ { INDEX_op_qemu_ld16s, { "r", "L" } },
		1124	+ { INDEX_op_qemu_ld32u, { "r", "L" } },
		1125	+ { INDEX_op_qemu_ld64, { "r", "r", "L" } },
		1126	+
		1127	+ { INDEX_op_qemu_st8, { "cb", "L" } },
		1128	+ { INDEX_op_qemu_st16, { "L", "L" } },
		1129	+ { INDEX_op_qemu_st32, { "L", "L" } },
		1130	+ { INDEX_op_qemu_st64, { "L", "L", "L" } },
		1131	+#else
		1132	+ { INDEX_op_qemu_ld8u, { "r", "L", "L" } },
		1133	+ { INDEX_op_qemu_ld8s, { "r", "L", "L" } },
		1134	+ { INDEX_op_qemu_ld16u, { "r", "L", "L" } },
		1135	+ { INDEX_op_qemu_ld16s, { "r", "L", "L" } },
		1136	+ { INDEX_op_qemu_ld32u, { "r", "L", "L" } },
		1137	+ { INDEX_op_qemu_ld64, { "r", "r", "L", "L" } },
		1138	+
		1139	+ { INDEX_op_qemu_st8, { "cb", "L", "L" } },
		1140	+ { INDEX_op_qemu_st16, { "L", "L", "L" } },
		1141	+ { INDEX_op_qemu_st32, { "L", "L", "L" } },
		1142	+ { INDEX_op_qemu_st64, { "L", "L", "L", "L" } },
		1143	+#endif
		1144	+ { -1 },
		1145	+};
		1146	+
		1147	+void tcg_target_init(TCGContext *s)
		1148	+{
		1149	+ /* fail safe */
		1150	+ if ((1 << CPU_TLB_ENTRY_BITS) != sizeof(CPUTLBEntry))
		1151	+ tcg_abort();
		1152	+
		1153	+ tcg_regset_set32(tcg_target_available_regs[TCG_TYPE_I32], 0, 0xff);
		1154	+ tcg_regset_set32(tcg_target_call_clobber_regs, 0,
		1155	+ (1 << TCG_REG_EAX) \|
		1156	+ (1 << TCG_REG_EDX) \|
		1157	+ (1 << TCG_REG_ECX));
		1158	+
		1159	+ tcg_regset_clear(s->reserved_regs);
		1160	+ tcg_regset_set_reg(s->reserved_regs, TCG_REG_ESP);
		1161	+
		1162	+ tcg_add_target_add_op_defs(x86_op_defs);
		1163	+}

tcg/i386/tcg-target.h 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#define TCG_TARGET_I386 1
		25	+
		26	+#define TCG_TARGET_REG_BITS 32
		27	+//#define TCG_TARGET_WORDS_BIGENDIAN
		28	+
		29	+#define TCG_TARGET_NB_REGS 8
		30	+
		31	+enum {
		32	+ TCG_REG_EAX = 0,
		33	+ TCG_REG_ECX,
		34	+ TCG_REG_EDX,
		35	+ TCG_REG_EBX,
		36	+ TCG_REG_ESP,
		37	+ TCG_REG_EBP,
		38	+ TCG_REG_ESI,
		39	+ TCG_REG_EDI,
		40	+};
		41	+
		42	+/* used for function call generation */
		43	+#define TCG_REG_CALL_STACK TCG_REG_ESP
		44	+#define TCG_TARGET_STACK_ALIGN 16
		45	+
		46	+/* Note: must be synced with dyngen-exec.h */
		47	+#define TCG_AREG0 TCG_REG_EBP
		48	+#define TCG_AREG1 TCG_REG_EBX
		49	+#define TCG_AREG2 TCG_REG_ESI
		50	+#define TCG_AREG3 TCG_REG_EDI
		51	+
		52	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		53	+{
		54	+}

tcg/tcg-dyngen.c 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#include <assert.h>
		25	+#include <stdarg.h>
		26	+#include <stdlib.h>
		27	+#include <stdio.h>
		28	+#include <string.h>
		29	+#include <inttypes.h>
		30	+
		31	+#include "config.h"
		32	+#include "osdep.h"
		33	+
		34	+#include "tcg.h"
		35	+
		36	+int __op_param1, __op_param2, __op_param3;
		37	+#if defined(__sparc__) \|\| defined(__arm__)
		38	+ void __op_gen_label1(){}
		39	+ void __op_gen_label2(){}
		40	+ void __op_gen_label3(){}
		41	+#else
		42	+ int __op_gen_label1, __op_gen_label2, __op_gen_label3;
		43	+#endif
		44	+int __op_jmp0, __op_jmp1, __op_jmp2, __op_jmp3;
		45	+
		46	+#if 0
		47	+#if defined(__s390__)
		48	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		49	+{
		50	+}
		51	+#elif defined(__ia64__)
		52	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		53	+{
		54	+ while (start < stop) {
		55	+ asm volatile ("fc %0" :: "r"(start));
		56	+ start += 32;
		57	+ }
		58	+ asm volatile (";;sync.i;;srlz.i;;");
		59	+}
		60	+#elif defined(__powerpc__)
		61	+
		62	+#define MIN_CACHE_LINE_SIZE 8 /* conservative value */
		63	+
		64	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		65	+{
		66	+ unsigned long p;
		67	+
		68	+ start &= ~(MIN_CACHE_LINE_SIZE - 1);
		69	+ stop = (stop + MIN_CACHE_LINE_SIZE - 1) & ~(MIN_CACHE_LINE_SIZE - 1);
		70	+
		71	+ for (p = start; p < stop; p += MIN_CACHE_LINE_SIZE) {
		72	+ asm volatile ("dcbst 0,%0" : : "r"(p) : "memory");
		73	+ }
		74	+ asm volatile ("sync" : : : "memory");
		75	+ for (p = start; p < stop; p += MIN_CACHE_LINE_SIZE) {
		76	+ asm volatile ("icbi 0,%0" : : "r"(p) : "memory");
		77	+ }
		78	+ asm volatile ("sync" : : : "memory");
		79	+ asm volatile ("isync" : : : "memory");
		80	+}
		81	+#elif defined(__alpha__)
		82	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		83	+{
		84	+ asm ("imb");
		85	+}
		86	+#elif defined(__sparc__)
		87	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		88	+{
		89	+ unsigned long p;
		90	+
		91	+ p = start & ~(8UL - 1UL);
		92	+ stop = (stop + (8UL - 1UL)) & ~(8UL - 1UL);
		93	+
		94	+ for (; p < stop; p += 8)
		95	+ __asm__ __volatile__("flush\t%0" : : "r" (p));
		96	+}
		97	+#elif defined(__arm__)
		98	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		99	+{
		100	+ register unsigned long _beg __asm ("a1") = start;
		101	+ register unsigned long _end __asm ("a2") = stop;
		102	+ register unsigned long _flg __asm ("a3") = 0;
		103	+ __asm __volatile__ ("swi 0x9f0002" : : "r" (_beg), "r" (_end), "r" (_flg));
		104	+}
		105	+#elif defined(__mc68000)
		106	+
		107	+# include <asm/cachectl.h>
		108	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		109	+{
		110	+ cacheflush(start,FLUSH_SCOPE_LINE,FLUSH_CACHE_BOTH,stop-start+16);
		111	+}
		112	+#elif defined(__mips__)
		113	+
		114	+#include <sys/cachectl.h>
		115	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		116	+{
		117	+ _flush_cache ((void *)start, stop - start, BCACHE);
		118	+}
		119	+#else
		120	+#error unsupported CPU
		121	+#endif
		122	+
		123	+#ifdef __alpha__
		124	+
		125	+register int gp asm("$29");
		126	+
		127	+static inline void immediate_ldah(void *p, int val) {
		128	+ uint32_t *dest = p;
		129	+ long high = ((val >> 16) + ((val >> 15) & 1)) & 0xffff;
		130	+
		131	+ *dest &= ~0xffff;
		132	+ *dest \|= high;
		133	+ *dest \|= 31 << 16;
		134	+}
		135	+static inline void immediate_lda(void *dest, int val) {
		136	+ (uint16_t ) dest = val;
		137	+}
		138	+void fix_bsr(void *p, int offset) {
		139	+ uint32_t *dest = p;
		140	+ *dest &= ~((1 << 21) - 1);
		141	+ *dest \|= (offset >> 2) & ((1 << 21) - 1);
		142	+}
		143	+
		144	+#endif /* __alpha__ */
		145	+
		146	+#ifdef __arm__
		147	+
		148	+#define ARM_LDR_TABLE_SIZE 1024
		149	+
		150	+typedef struct LDREntry {
		151	+ uint8_t *ptr;
		152	+ uint32_t *data_ptr;
		153	+ unsigned type:2;
		154	+} LDREntry;
		155	+
		156	+static LDREntry arm_ldr_table[1024];
		157	+static uint32_t arm_data_table[ARM_LDR_TABLE_SIZE];
		158	+
		159	+extern char exec_loop;
		160	+
		161	+static inline void arm_reloc_pc24(uint32_t *ptr, uint32_t insn, int val)
		162	+{
		163	+ *ptr = (insn & ~0xffffff) \| ((insn + ((val - (int)ptr) >> 2)) & 0xffffff);
		164	+}
		165	+
		166	+static uint8_t arm_flush_ldr(uint8_t gen_code_ptr,
		167	+ LDREntry ldr_start, LDREntry ldr_end,
		168	+ uint32_t data_start, uint32_t data_end,
		169	+ int gen_jmp)
		170	+{
		171	+ LDREntry *le;
		172	+ uint32_t *ptr;
		173	+ int offset, data_size, target;
		174	+ uint8_t *data_ptr;
		175	+ uint32_t insn;
		176	+ uint32_t mask;
		177	+
		178	+ data_size = (data_end - data_start) << 2;
		179	+
		180	+ if (gen_jmp) {
		181	+ /* generate branch to skip the data */
		182	+ if (data_size == 0)
		183	+ return gen_code_ptr;
		184	+ target = (long)gen_code_ptr + data_size + 4;
		185	+ arm_reloc_pc24((uint32_t *)gen_code_ptr, 0xeafffffe, target);
		186	+ gen_code_ptr += 4;
		187	+ }
		188	+
		189	+ /* copy the data */
		190	+ data_ptr = gen_code_ptr;
		191	+ memcpy(gen_code_ptr, data_start, data_size);
		192	+ gen_code_ptr += data_size;
		193	+
		194	+ /* patch the ldr to point to the data */
		195	+ for(le = ldr_start; le < ldr_end; le++) {
		196	+ ptr = (uint32_t *)le->ptr;
		197	+ offset = ((unsigned long)(le->data_ptr) - (unsigned long)data_start) +
		198	+ (unsigned long)data_ptr -
		199	+ (unsigned long)ptr - 8;
		200	+ if (offset < 0) {
		201	+ fprintf(stderr, "Negative constant pool offset\n");
		202	+ tcg_abort();
		203	+ }
		204	+ switch (le->type) {
		205	+ case 0: /* ldr */
		206	+ mask = ~0x00800fff;
		207	+ if (offset >= 4096) {
		208	+ fprintf(stderr, "Bad ldr offset\n");
		209	+ tcg_abort();
		210	+ }
		211	+ break;
		212	+ case 1: /* ldc */
		213	+ mask = ~0x008000ff;
		214	+ if (offset >= 1024 ) {
		215	+ fprintf(stderr, "Bad ldc offset\n");
		216	+ tcg_abort();
		217	+ }
		218	+ break;
		219	+ case 2: /* add */
		220	+ mask = ~0xfff;
		221	+ if (offset >= 1024 ) {
		222	+ fprintf(stderr, "Bad add offset\n");
		223	+ tcg_abort();
		224	+ }
		225	+ break;
		226	+ default:
		227	+ fprintf(stderr, "Bad pc relative fixup\n");
		228	+ tcg_abort();
		229	+ }
		230	+ insn = *ptr & mask;
		231	+ switch (le->type) {
		232	+ case 0: /* ldr */
		233	+ insn \|= offset \| 0x00800000;
		234	+ break;
		235	+ case 1: /* ldc */
		236	+ insn \|= (offset >> 2) \| 0x00800000;
		237	+ break;
		238	+ case 2: /* add */
		239	+ insn \|= (offset >> 2) \| 0xf00;
		240	+ break;
		241	+ }
		242	+ *ptr = insn;
		243	+ }
		244	+ return gen_code_ptr;
		245	+}
		246	+
		247	+#endif /* __arm__ */
		248	+
		249	+#ifdef __ia64
		250	+
		251	+/* Patch instruction with "val" where "mask" has 1 bits. */
		252	+static inline void ia64_patch (uint64_t insn_addr, uint64_t mask, uint64_t val)
		253	+{
		254	+ uint64_t m0, m1, v0, v1, b0, b1, b = (uint64_t ) (insn_addr & -16);
		255	+# define insn_mask ((1UL << 41) - 1)
		256	+ unsigned long shift;
		257	+
		258	+ b0 = b[0]; b1 = b[1];
		259	+ shift = 5 + 41 * (insn_addr % 16); /* 5 template, 3 x 41-bit insns */
		260	+ if (shift >= 64) {
		261	+ m1 = mask << (shift - 64);
		262	+ v1 = val << (shift - 64);
		263	+ } else {
		264	+ m0 = mask << shift; m1 = mask >> (64 - shift);
		265	+ v0 = val << shift; v1 = val >> (64 - shift);
		266	+ b[0] = (b0 & ~m0) \| (v0 & m0);
		267	+ }
		268	+ b[1] = (b1 & ~m1) \| (v1 & m1);
		269	+}
		270	+
		271	+static inline void ia64_patch_imm60 (uint64_t insn_addr, uint64_t val)
		272	+{
		273	+ ia64_patch(insn_addr,
		274	+ 0x011ffffe000UL,
		275	+ ( ((val & 0x0800000000000000UL) >> 23) /* bit 59 -> 36 */
		276	+ \| ((val & 0x00000000000fffffUL) << 13) /* bit 0 -> 13 */));
		277	+ ia64_patch(insn_addr - 1, 0x1fffffffffcUL, val >> 18);
		278	+}
		279	+
		280	+static inline void ia64_imm64 (void *insn, uint64_t val)
		281	+{
		282	+ /* Ignore the slot number of the relocation; GCC and Intel
		283	+ toolchains differed for some time on whether IMM64 relocs are
		284	+ against slot 1 (Intel) or slot 2 (GCC). */
		285	+ uint64_t insn_addr = (uint64_t) insn & ~3UL;
		286	+
		287	+ ia64_patch(insn_addr + 2,
		288	+ 0x01fffefe000UL,
		289	+ ( ((val & 0x8000000000000000UL) >> 27) /* bit 63 -> 36 */
		290	+ \| ((val & 0x0000000000200000UL) << 0) /* bit 21 -> 21 */
		291	+ \| ((val & 0x00000000001f0000UL) << 6) /* bit 16 -> 22 */
		292	+ \| ((val & 0x000000000000ff80UL) << 20) /* bit 7 -> 27 */
		293	+ \| ((val & 0x000000000000007fUL) << 13) /* bit 0 -> 13 */)
		294	+ );
		295	+ ia64_patch(insn_addr + 1, 0x1ffffffffffUL, val >> 22);
		296	+}
		297	+
		298	+static inline void ia64_imm60b (void *insn, uint64_t val)
		299	+{
		300	+ /* Ignore the slot number of the relocation; GCC and Intel
		301	+ toolchains differed for some time on whether IMM64 relocs are
		302	+ against slot 1 (Intel) or slot 2 (GCC). */
		303	+ uint64_t insn_addr = (uint64_t) insn & ~3UL;
		304	+
		305	+ if (val + ((uint64_t) 1 << 59) >= (1UL << 60))
		306	+ fprintf(stderr, "%s: value %ld out of IMM60 range\n",
		307	+ __FUNCTION__, (int64_t) val);
		308	+ ia64_patch_imm60(insn_addr + 2, val);
		309	+}
		310	+
		311	+static inline void ia64_imm22 (void *insn, uint64_t val)
		312	+{
		313	+ if (val + (1 << 21) >= (1 << 22))
		314	+ fprintf(stderr, "%s: value %li out of IMM22 range\n",
		315	+ __FUNCTION__, (int64_t)val);
		316	+ ia64_patch((uint64_t) insn, 0x01fffcfe000UL,
		317	+ ( ((val & 0x200000UL) << 15) /* bit 21 -> 36 */
		318	+ \| ((val & 0x1f0000UL) << 6) /* bit 16 -> 22 */
		319	+ \| ((val & 0x00ff80UL) << 20) /* bit 7 -> 27 */
		320	+ \| ((val & 0x00007fUL) << 13) /* bit 0 -> 13 */));
		321	+}
		322	+
		323	+/* Like ia64_imm22(), but also clear bits 20-21. For addl, this has
		324	+ the effect of turning "addl rX=imm22,rY" into "addl
		325	+ rX=imm22,r0". */
		326	+static inline void ia64_imm22_r0 (void *insn, uint64_t val)
		327	+{
		328	+ if (val + (1 << 21) >= (1 << 22))
		329	+ fprintf(stderr, "%s: value %li out of IMM22 range\n",
		330	+ __FUNCTION__, (int64_t)val);
		331	+ ia64_patch((uint64_t) insn, 0x01fffcfe000UL \| (0x3UL << 20),
		332	+ ( ((val & 0x200000UL) << 15) /* bit 21 -> 36 */
		333	+ \| ((val & 0x1f0000UL) << 6) /* bit 16 -> 22 */
		334	+ \| ((val & 0x00ff80UL) << 20) /* bit 7 -> 27 */
		335	+ \| ((val & 0x00007fUL) << 13) /* bit 0 -> 13 */));
		336	+}
		337	+
		338	+static inline void ia64_imm21b (void *insn, uint64_t val)
		339	+{
		340	+ if (val + (1 << 20) >= (1 << 21))
		341	+ fprintf(stderr, "%s: value %li out of IMM21b range\n",
		342	+ __FUNCTION__, (int64_t)val);
		343	+ ia64_patch((uint64_t) insn, 0x11ffffe000UL,
		344	+ ( ((val & 0x100000UL) << 16) /* bit 20 -> 36 */
		345	+ \| ((val & 0x0fffffUL) << 13) /* bit 0 -> 13 */));
		346	+}
		347	+
		348	+static inline void ia64_nop_b (void *insn)
		349	+{
		350	+ ia64_patch((uint64_t) insn, (1UL << 41) - 1, 2UL << 37);
		351	+}
		352	+
		353	+static inline void ia64_ldxmov(void *insn, uint64_t val)
		354	+{
		355	+ if (val + (1 << 21) < (1 << 22))
		356	+ ia64_patch((uint64_t) insn, 0x1fff80fe000UL, 8UL << 37);
		357	+}
		358	+
		359	+static inline int ia64_patch_ltoff(void *insn, uint64_t val,
		360	+ int relaxable)
		361	+{
		362	+ if (relaxable && (val + (1 << 21) < (1 << 22))) {
		363	+ ia64_imm22_r0(insn, val);
		364	+ return 0;
		365	+ }
		366	+ return 1;
		367	+}
		368	+
		369	+struct ia64_fixup {
		370	+ struct ia64_fixup *next;
		371	+ void addr; / address that needs to be patched */
		372	+ long value;
		373	+};
		374	+
		375	+#define IA64_PLT(insn, plt_index) \
		376	+do { \
		377	+ struct ia64_fixup fixup = alloca(sizeof(fixup)); \
		378	+ fixup->next = plt_fixes; \
		379	+ plt_fixes = fixup; \
		380	+ fixup->addr = (insn); \
		381	+ fixup->value = (plt_index); \
		382	+ plt_offset[(plt_index)] = 1; \
		383	+} while (0)
		384	+
		385	+#define IA64_LTOFF(insn, val, relaxable) \
		386	+do { \
		387	+ if (ia64_patch_ltoff(insn, val, relaxable)) { \
		388	+ struct ia64_fixup fixup = alloca(sizeof(fixup)); \
		389	+ fixup->next = ltoff_fixes; \
		390	+ ltoff_fixes = fixup; \
		391	+ fixup->addr = (insn); \
		392	+ fixup->value = (val); \
		393	+ } \
		394	+} while (0)
		395	+
		396	+static inline void ia64_apply_fixes (uint8_t **gen_code_pp,
		397	+ struct ia64_fixup *ltoff_fixes,
		398	+ uint64_t gp,
		399	+ struct ia64_fixup *plt_fixes,
		400	+ int num_plts,
		401	+ unsigned long *plt_target,
		402	+ unsigned int *plt_offset)
		403	+{
		404	+ static const uint8_t plt_bundle[] = {
		405	+ 0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, /* nop 0; movl r1=GP */
		406	+ 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00, 0x60,
		407	+
		408	+ 0x05, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, /* nop 0; brl IP */
		409	+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0
		410	+ };
		411	+ uint8_t gen_code_ptr = gen_code_pp, plt_start, got_start;
		412	+ uint64_t *vp;
		413	+ struct ia64_fixup *fixup;
		414	+ unsigned int offset = 0;
		415	+ struct fdesc {
		416	+ long ip;
		417	+ long gp;
		418	+ } *fdesc;
		419	+ int i;
		420	+
		421	+ if (plt_fixes) {
		422	+ plt_start = gen_code_ptr;
		423	+
		424	+ for (i = 0; i < num_plts; ++i) {
		425	+ if (plt_offset[i]) {
		426	+ plt_offset[i] = offset;
		427	+ offset += sizeof(plt_bundle);
		428	+
		429	+ fdesc = (struct fdesc *) plt_target[i];
		430	+ memcpy(gen_code_ptr, plt_bundle, sizeof(plt_bundle));
		431	+ ia64_imm64 (gen_code_ptr + 0x02, fdesc->gp);
		432	+ ia64_imm60b(gen_code_ptr + 0x12,
		433	+ (fdesc->ip - (long) (gen_code_ptr + 0x10)) >> 4);
		434	+ gen_code_ptr += sizeof(plt_bundle);
		435	+ }
		436	+ }
		437	+
		438	+ for (fixup = plt_fixes; fixup; fixup = fixup->next)
		439	+ ia64_imm21b(fixup->addr,
		440	+ ((long) plt_start + plt_offset[fixup->value]
		441	+ - ((long) fixup->addr & ~0xf)) >> 4);
		442	+ }
		443	+
		444	+ got_start = gen_code_ptr;
		445	+
		446	+ /* First, create the GOT: */
		447	+ for (fixup = ltoff_fixes; fixup; fixup = fixup->next) {
		448	+ /* first check if we already have this value in the GOT: */
		449	+ for (vp = (uint64_t ) got_start; vp < (uint64_t ) gen_code_ptr; ++vp)
		450	+ if (*vp == fixup->value)
		451	+ break;
		452	+ if (vp == (uint64_t *) gen_code_ptr) {
		453	+ /* Nope, we need to put the value in the GOT: */
		454	+ *vp = fixup->value;
		455	+ gen_code_ptr += 8;
		456	+ }
		457	+ ia64_imm22(fixup->addr, (long) vp - gp);
		458	+ }
		459	+ /* Keep code ptr aligned. */
		460	+ if ((long) gen_code_ptr & 15)
		461	+ gen_code_ptr += 8;
		462	+ *gen_code_pp = gen_code_ptr;
		463	+}
		464	+#endif
		465	+#endif
		466	+
		467	+const TCGArg dyngen_op(TCGContext s, int opc, const TCGArg *opparam_ptr)
		468	+{
		469	+ uint8_t *gen_code_ptr;
		470	+
		471	+ gen_code_ptr = s->code_ptr;
		472	+ switch(opc) {
		473	+
		474	+/* op.h is dynamically generated by dyngen.c from op.c */
		475	+#include "op.h"
		476	+
		477	+ default:
		478	+ tcg_abort();
		479	+ }
		480	+ s->code_ptr = gen_code_ptr;
		481	+ return opparam_ptr;
		482	+}

tcg/tcg-op.h 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#include "tcg.h"
		25	+
		26	+/* legacy dyngen operations */
		27	+#include "gen-op.h"
		28	+
		29	+int gen_new_label(void);
		30	+
		31	+static inline void tcg_gen_op1(int opc, TCGArg arg1)
		32	+{
		33	+ *gen_opc_ptr++ = opc;
		34	+ *gen_opparam_ptr++ = arg1;
		35	+}
		36	+
		37	+static inline void tcg_gen_op2(int opc, TCGArg arg1, TCGArg arg2)
		38	+{
		39	+ *gen_opc_ptr++ = opc;
		40	+ *gen_opparam_ptr++ = arg1;
		41	+ *gen_opparam_ptr++ = arg2;
		42	+}
		43	+
		44	+static inline void tcg_gen_op3(int opc, TCGArg arg1, TCGArg arg2, TCGArg arg3)
		45	+{
		46	+ *gen_opc_ptr++ = opc;
		47	+ *gen_opparam_ptr++ = arg1;
		48	+ *gen_opparam_ptr++ = arg2;
		49	+ *gen_opparam_ptr++ = arg3;
		50	+}
		51	+
		52	+static inline void tcg_gen_op4(int opc, TCGArg arg1, TCGArg arg2, TCGArg arg3,
		53	+ TCGArg arg4)
		54	+{
		55	+ *gen_opc_ptr++ = opc;
		56	+ *gen_opparam_ptr++ = arg1;
		57	+ *gen_opparam_ptr++ = arg2;
		58	+ *gen_opparam_ptr++ = arg3;
		59	+ *gen_opparam_ptr++ = arg4;
		60	+}
		61	+
		62	+static inline void tcg_gen_op5(int opc, TCGArg arg1, TCGArg arg2,
		63	+ TCGArg arg3, TCGArg arg4,
		64	+ TCGArg arg5)
		65	+{
		66	+ *gen_opc_ptr++ = opc;
		67	+ *gen_opparam_ptr++ = arg1;
		68	+ *gen_opparam_ptr++ = arg2;
		69	+ *gen_opparam_ptr++ = arg3;
		70	+ *gen_opparam_ptr++ = arg4;
		71	+ *gen_opparam_ptr++ = arg5;
		72	+}
		73	+
		74	+static inline void tcg_gen_op6(int opc, TCGArg arg1, TCGArg arg2,
		75	+ TCGArg arg3, TCGArg arg4,
		76	+ TCGArg arg5, TCGArg arg6)
		77	+{
		78	+ *gen_opc_ptr++ = opc;
		79	+ *gen_opparam_ptr++ = arg1;
		80	+ *gen_opparam_ptr++ = arg2;
		81	+ *gen_opparam_ptr++ = arg3;
		82	+ *gen_opparam_ptr++ = arg4;
		83	+ *gen_opparam_ptr++ = arg5;
		84	+ *gen_opparam_ptr++ = arg6;
		85	+}
		86	+
		87	+static inline void gen_set_label(int n)
		88	+{
		89	+ tcg_gen_op1(INDEX_op_set_label, n);
		90	+}
		91	+
		92	+static inline void tcg_gen_mov_i32(int ret, int arg)
		93	+{
		94	+ tcg_gen_op2(INDEX_op_mov_i32, ret, arg);
		95	+}
		96	+
		97	+static inline void tcg_gen_movi_i32(int ret, int32_t arg)
		98	+{
		99	+ tcg_gen_op2(INDEX_op_movi_i32, ret, arg);
		100	+}
		101	+
		102	+/* helper calls */
		103	+#define TCG_HELPER_CALL_FLAGS 0
		104	+
		105	+static inline void tcg_gen_helper_0_0(void *func)
		106	+{
		107	+ tcg_gen_call(&tcg_ctx,
		108	+ tcg_const_ptr((tcg_target_long)func), TCG_HELPER_CALL_FLAGS,
		109	+ 0, NULL, 0, NULL);
		110	+}
		111	+
		112	+static inline void tcg_gen_helper_0_1(void *func, TCGArg arg)
		113	+{
		114	+ tcg_gen_call(&tcg_ctx,
		115	+ tcg_const_ptr((tcg_target_long)func), TCG_HELPER_CALL_FLAGS,
		116	+ 0, NULL, 1, &arg);
		117	+}
		118	+
		119	+static inline void tcg_gen_helper_0_2(void *func, TCGArg arg1, TCGArg arg2)
		120	+{
		121	+ TCGArg args[2];
		122	+ args[0] = arg1;
		123	+ args[1] = arg2;
		124	+ tcg_gen_call(&tcg_ctx,
		125	+ tcg_const_ptr((tcg_target_long)func), TCG_HELPER_CALL_FLAGS,
		126	+ 0, NULL, 2, args);
		127	+}
		128	+
		129	+static inline void tcg_gen_helper_1_2(void *func, TCGArg ret,
		130	+ TCGArg arg1, TCGArg arg2)
		131	+{
		132	+ TCGArg args[2];
		133	+ args[0] = arg1;
		134	+ args[1] = arg2;
		135	+ tcg_gen_call(&tcg_ctx,
		136	+ tcg_const_ptr((tcg_target_long)func), TCG_HELPER_CALL_FLAGS,
		137	+ 1, &ret, 2, args);
		138	+}
		139	+
		140	+/* 32 bit ops */
		141	+
		142	+static inline void tcg_gen_ld8u_i32(int ret, int arg2, tcg_target_long offset)
		143	+{
		144	+ tcg_gen_op3(INDEX_op_ld8u_i32, ret, arg2, offset);
		145	+}
		146	+
		147	+static inline void tcg_gen_ld8s_i32(int ret, int arg2, tcg_target_long offset)
		148	+{
		149	+ tcg_gen_op3(INDEX_op_ld8s_i32, ret, arg2, offset);
		150	+}
		151	+
		152	+static inline void tcg_gen_ld16u_i32(int ret, int arg2, tcg_target_long offset)
		153	+{
		154	+ tcg_gen_op3(INDEX_op_ld16u_i32, ret, arg2, offset);
		155	+}
		156	+
		157	+static inline void tcg_gen_ld16s_i32(int ret, int arg2, tcg_target_long offset)
		158	+{
		159	+ tcg_gen_op3(INDEX_op_ld16s_i32, ret, arg2, offset);
		160	+}
		161	+
		162	+static inline void tcg_gen_ld_i32(int ret, int arg2, tcg_target_long offset)
		163	+{
		164	+ tcg_gen_op3(INDEX_op_ld_i32, ret, arg2, offset);
		165	+}
		166	+
		167	+static inline void tcg_gen_st8_i32(int arg1, int arg2, tcg_target_long offset)
		168	+{
		169	+ tcg_gen_op3(INDEX_op_st8_i32, arg1, arg2, offset);
		170	+}
		171	+
		172	+static inline void tcg_gen_st16_i32(int arg1, int arg2, tcg_target_long offset)
		173	+{
		174	+ tcg_gen_op3(INDEX_op_st16_i32, arg1, arg2, offset);
		175	+}
		176	+
		177	+static inline void tcg_gen_st_i32(int arg1, int arg2, tcg_target_long offset)
		178	+{
		179	+ tcg_gen_op3(INDEX_op_st_i32, arg1, arg2, offset);
		180	+}
		181	+
		182	+static inline void tcg_gen_add_i32(int ret, int arg1, int arg2)
		183	+{
		184	+ tcg_gen_op3(INDEX_op_add_i32, ret, arg1, arg2);
		185	+}
		186	+
		187	+static inline void tcg_gen_addi_i32(int ret, int arg1, int32_t arg2)
		188	+{
		189	+ tcg_gen_add_i32(ret, arg1, tcg_const_i32(arg2));
		190	+}
		191	+
		192	+static inline void tcg_gen_sub_i32(int ret, int arg1, int arg2)
		193	+{
		194	+ tcg_gen_op3(INDEX_op_sub_i32, ret, arg1, arg2);
		195	+}
		196	+
		197	+static inline void tcg_gen_subi_i32(int ret, int arg1, int32_t arg2)
		198	+{
		199	+ tcg_gen_sub_i32(ret, arg1, tcg_const_i32(arg2));
		200	+}
		201	+
		202	+static inline void tcg_gen_and_i32(int ret, int arg1, int arg2)
		203	+{
		204	+ tcg_gen_op3(INDEX_op_and_i32, ret, arg1, arg2);
		205	+}
		206	+
		207	+static inline void tcg_gen_andi_i32(int ret, int arg1, int32_t arg2)
		208	+{
		209	+ /* some cases can be optimized here */
		210	+ if (arg2 == 0) {
		211	+ tcg_gen_movi_i32(ret, 0);
		212	+ } else if (arg2 == 0xffffffff) {
		213	+ tcg_gen_mov_i32(ret, arg1);
		214	+ } else {
		215	+ tcg_gen_and_i32(ret, arg1, tcg_const_i32(arg2));
		216	+ }
		217	+}
		218	+
		219	+static inline void tcg_gen_or_i32(int ret, int arg1, int arg2)
		220	+{
		221	+ tcg_gen_op3(INDEX_op_or_i32, ret, arg1, arg2);
		222	+}
		223	+
		224	+static inline void tcg_gen_ori_i32(int ret, int arg1, int32_t arg2)
		225	+{
		226	+ /* some cases can be optimized here */
		227	+ if (arg2 == 0xffffffff) {
		228	+ tcg_gen_movi_i32(ret, 0);
		229	+ } else if (arg2 == 0) {
		230	+ tcg_gen_mov_i32(ret, arg1);
		231	+ } else {
		232	+ tcg_gen_or_i32(ret, arg1, tcg_const_i32(arg2));
		233	+ }
		234	+}
		235	+
		236	+static inline void tcg_gen_xor_i32(int ret, int arg1, int arg2)
		237	+{
		238	+ tcg_gen_op3(INDEX_op_xor_i32, ret, arg1, arg2);
		239	+}
		240	+
		241	+static inline void tcg_gen_xori_i32(int ret, int arg1, int32_t arg2)
		242	+{
		243	+ /* some cases can be optimized here */
		244	+ if (arg2 == 0) {
		245	+ tcg_gen_mov_i32(ret, arg1);
		246	+ } else {
		247	+ tcg_gen_xor_i32(ret, arg1, tcg_const_i32(arg2));
		248	+ }
		249	+}
		250	+
		251	+static inline void tcg_gen_shl_i32(int ret, int arg1, int arg2)
		252	+{
		253	+ tcg_gen_op3(INDEX_op_shl_i32, ret, arg1, arg2);
		254	+}
		255	+
		256	+static inline void tcg_gen_shli_i32(int ret, int arg1, int32_t arg2)
		257	+{
		258	+ tcg_gen_shl_i32(ret, arg1, tcg_const_i32(arg2));
		259	+}
		260	+
		261	+static inline void tcg_gen_shr_i32(int ret, int arg1, int arg2)
		262	+{
		263	+ tcg_gen_op3(INDEX_op_shr_i32, ret, arg1, arg2);
		264	+}
		265	+
		266	+static inline void tcg_gen_shri_i32(int ret, int arg1, int32_t arg2)
		267	+{
		268	+ tcg_gen_shr_i32(ret, arg1, tcg_const_i32(arg2));
		269	+}
		270	+
		271	+static inline void tcg_gen_sar_i32(int ret, int arg1, int arg2)
		272	+{
		273	+ tcg_gen_op3(INDEX_op_sar_i32, ret, arg1, arg2);
		274	+}
		275	+
		276	+static inline void tcg_gen_sari_i32(int ret, int arg1, int32_t arg2)
		277	+{
		278	+ tcg_gen_sar_i32(ret, arg1, tcg_const_i32(arg2));
		279	+}
		280	+
		281	+static inline void tcg_gen_brcond_i32(int cond, TCGArg arg1, TCGArg arg2,
		282	+ int label_index)
		283	+{
		284	+ tcg_gen_op4(INDEX_op_brcond_i32, arg1, arg2, cond, label_index);
		285	+}
		286	+
		287	+static inline void tcg_gen_mul_i32(int ret, int arg1, int arg2)
		288	+{
		289	+ tcg_gen_op3(INDEX_op_mul_i32, ret, arg1, arg2);
		290	+}
		291	+
		292	+#ifdef TCG_TARGET_HAS_div_i32
		293	+static inline void tcg_gen_div_i32(int ret, int arg1, int arg2)
		294	+{
		295	+ tcg_gen_op3(INDEX_op_div_i32, ret, arg1, arg2);
		296	+}
		297	+
		298	+static inline void tcg_gen_rem_i32(int ret, int arg1, int arg2)
		299	+{
		300	+ tcg_gen_op3(INDEX_op_rem_i32, ret, arg1, arg2);
		301	+}
		302	+
		303	+static inline void tcg_gen_divu_i32(int ret, int arg1, int arg2)
		304	+{
		305	+ tcg_gen_op3(INDEX_op_divu_i32, ret, arg1, arg2);
		306	+}
		307	+
		308	+static inline void tcg_gen_remu_i32(int ret, int arg1, int arg2)
		309	+{
		310	+ tcg_gen_op3(INDEX_op_remu_i32, ret, arg1, arg2);
		311	+}
		312	+#else
		313	+static inline void tcg_gen_div_i32(int ret, int arg1, int arg2)
		314	+{
		315	+ int t0;
		316	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		317	+ tcg_gen_sari_i32(t0, arg1, 31);
		318	+ tcg_gen_op5(INDEX_op_div2_i32, ret, t0, arg1, t0, arg2);
		319	+}
		320	+
		321	+static inline void tcg_gen_rem_i32(int ret, int arg1, int arg2)
		322	+{
		323	+ int t0;
		324	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		325	+ tcg_gen_sari_i32(t0, arg1, 31);
		326	+ tcg_gen_op5(INDEX_op_div2_i32, t0, ret, arg1, t0, arg2);
		327	+}
		328	+
		329	+static inline void tcg_gen_divu_i32(int ret, int arg1, int arg2)
		330	+{
		331	+ int t0;
		332	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		333	+ tcg_gen_movi_i32(t0, 0);
		334	+ tcg_gen_op5(INDEX_op_divu2_i32, ret, t0, arg1, t0, arg2);
		335	+}
		336	+
		337	+static inline void tcg_gen_remu_i32(int ret, int arg1, int arg2)
		338	+{
		339	+ int t0;
		340	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		341	+ tcg_gen_movi_i32(t0, 0);
		342	+ tcg_gen_op5(INDEX_op_divu2_i32, t0, ret, arg1, t0, arg2);
		343	+}
		344	+#endif
		345	+
		346	+#if TCG_TARGET_REG_BITS == 32
		347	+
		348	+static inline void tcg_gen_mov_i64(int ret, int arg)
		349	+{
		350	+ tcg_gen_mov_i32(ret, arg);
		351	+ tcg_gen_mov_i32(ret + 1, arg + 1);
		352	+}
		353	+
		354	+static inline void tcg_gen_movi_i64(int ret, int64_t arg)
		355	+{
		356	+ tcg_gen_movi_i32(ret, arg);
		357	+ tcg_gen_movi_i32(ret + 1, arg >> 32);
		358	+}
		359	+
		360	+static inline void tcg_gen_ld8u_i64(int ret, int arg2, tcg_target_long offset)
		361	+{
		362	+ tcg_gen_ld8u_i32(ret, arg2, offset);
		363	+ tcg_gen_movi_i32(ret + 1, 0);
		364	+}
		365	+
		366	+static inline void tcg_gen_ld8s_i64(int ret, int arg2, tcg_target_long offset)
		367	+{
		368	+ tcg_gen_ld8s_i32(ret, arg2, offset);
		369	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		370	+}
		371	+
		372	+static inline void tcg_gen_ld16u_i64(int ret, int arg2, tcg_target_long offset)
		373	+{
		374	+ tcg_gen_ld16u_i32(ret, arg2, offset);
		375	+ tcg_gen_movi_i32(ret + 1, 0);
		376	+}
		377	+
		378	+static inline void tcg_gen_ld16s_i64(int ret, int arg2, tcg_target_long offset)
		379	+{
		380	+ tcg_gen_ld16s_i32(ret, arg2, offset);
		381	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		382	+}
		383	+
		384	+static inline void tcg_gen_ld32u_i64(int ret, int arg2, tcg_target_long offset)
		385	+{
		386	+ tcg_gen_ld_i32(ret, arg2, offset);
		387	+ tcg_gen_movi_i32(ret + 1, 0);
		388	+}
		389	+
		390	+static inline void tcg_gen_ld32s_i64(int ret, int arg2, tcg_target_long offset)
		391	+{
		392	+ tcg_gen_ld_i32(ret, arg2, offset);
		393	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		394	+}
		395	+
		396	+static inline void tcg_gen_ld_i64(int ret, int arg2, tcg_target_long offset)
		397	+{
		398	+ /* since arg2 and ret have different types, they cannot be the
		399	+ same temporary */
		400	+#ifdef TCG_TARGET_WORDS_BIGENDIAN
		401	+ tcg_gen_ld_i32(ret + 1, arg2, offset);
		402	+ tcg_gen_ld_i32(ret, arg2, offset + 4);
		403	+#else
		404	+ tcg_gen_ld_i32(ret, arg2, offset);
		405	+ tcg_gen_ld_i32(ret + 1, arg2, offset + 4);
		406	+#endif
		407	+}
		408	+
		409	+static inline void tcg_gen_st8_i64(int arg1, int arg2, tcg_target_long offset)
		410	+{
		411	+ tcg_gen_st8_i32(arg1, arg2, offset);
		412	+}
		413	+
		414	+static inline void tcg_gen_st16_i64(int arg1, int arg2, tcg_target_long offset)
		415	+{
		416	+ tcg_gen_st16_i32(arg1, arg2, offset);
		417	+}
		418	+
		419	+static inline void tcg_gen_st32_i64(int arg1, int arg2, tcg_target_long offset)
		420	+{
		421	+ tcg_gen_st_i32(arg1, arg2, offset);
		422	+}
		423	+
		424	+static inline void tcg_gen_st_i64(int arg1, int arg2, tcg_target_long offset)
		425	+{
		426	+#ifdef TCG_TARGET_WORDS_BIGENDIAN
		427	+ tcg_gen_st_i32(arg1 + 1, arg2, offset);
		428	+ tcg_gen_st_i32(arg1, arg2, offset + 4);
		429	+#else
		430	+ tcg_gen_st_i32(arg1, arg2, offset);
		431	+ tcg_gen_st_i32(arg1 + 1, arg2, offset + 4);
		432	+#endif
		433	+}
		434	+
		435	+static inline void tcg_gen_add_i64(int ret, int arg1, int arg2)
		436	+{
		437	+ tcg_gen_op6(INDEX_op_add2_i32, ret, ret + 1,
		438	+ arg1, arg1 + 1, arg2, arg2 + 1);
		439	+}
		440	+
		441	+static inline void tcg_gen_addi_i64(int ret, int arg1, int64_t arg2)
		442	+{
		443	+ tcg_gen_add_i64(ret, arg1, tcg_const_i64(arg2));
		444	+}
		445	+
		446	+static inline void tcg_gen_sub_i64(int ret, int arg1, int arg2)
		447	+{
		448	+ tcg_gen_op6(INDEX_op_sub2_i32, ret, ret + 1,
		449	+ arg1, arg1 + 1, arg2, arg2 + 1);
		450	+}
		451	+
		452	+static inline void tcg_gen_subi_i64(int ret, int arg1, int64_t arg2)
		453	+{
		454	+ tcg_gen_sub_i64(ret, arg1, tcg_const_i64(arg2));
		455	+}
		456	+
		457	+static inline void tcg_gen_and_i64(int ret, int arg1, int arg2)
		458	+{
		459	+ tcg_gen_and_i32(ret, arg1, arg2);
		460	+ tcg_gen_and_i32(ret + 1, arg1 + 1, arg2 + 1);
		461	+}
		462	+
		463	+static inline void tcg_gen_andi_i64(int ret, int arg1, int64_t arg2)
		464	+{
		465	+ tcg_gen_andi_i32(ret, arg1, arg2);
		466	+ tcg_gen_andi_i32(ret + 1, arg1 + 1, arg2 >> 32);
		467	+}
		468	+
		469	+static inline void tcg_gen_or_i64(int ret, int arg1, int arg2)
		470	+{
		471	+ tcg_gen_or_i32(ret, arg1, arg2);
		472	+ tcg_gen_or_i32(ret + 1, arg1 + 1, arg2 + 1);
		473	+}
		474	+
		475	+static inline void tcg_gen_ori_i64(int ret, int arg1, int64_t arg2)
		476	+{
		477	+ tcg_gen_ori_i32(ret, arg1, arg2);
		478	+ tcg_gen_ori_i32(ret + 1, arg1 + 1, arg2 >> 32);
		479	+}
		480	+
		481	+static inline void tcg_gen_xor_i64(int ret, int arg1, int arg2)
		482	+{
		483	+ tcg_gen_xor_i32(ret, arg1, arg2);
		484	+ tcg_gen_xor_i32(ret + 1, arg1 + 1, arg2 + 1);
		485	+}
		486	+
		487	+static inline void tcg_gen_xori_i64(int ret, int arg1, int64_t arg2)
		488	+{
		489	+ tcg_gen_xori_i32(ret, arg1, arg2);
		490	+ tcg_gen_xori_i32(ret + 1, arg1 + 1, arg2 >> 32);
		491	+}
		492	+
		493	+/* XXX: use generic code when basic block handling is OK or CPU
		494	+ specific code (x86) */
		495	+static inline void tcg_gen_shl_i64(int ret, int arg1, int64_t arg2)
		496	+{
		497	+ tcg_gen_helper_1_2(tcg_helper_shl_i64, ret, arg1, arg2);
		498	+}
		499	+
		500	+static inline void tcg_gen_shli_i64(int ret, int arg1, int64_t arg2)
		501	+{
		502	+ tcg_gen_shifti_i64(ret, arg1, arg2, 0, 0);
		503	+}
		504	+
		505	+static inline void tcg_gen_shr_i64(int ret, int arg1, int64_t arg2)
		506	+{
		507	+ tcg_gen_helper_1_2(tcg_helper_shr_i64, ret, arg1, arg2);
		508	+}
		509	+
		510	+static inline void tcg_gen_shri_i64(int ret, int arg1, int64_t arg2)
		511	+{
		512	+ tcg_gen_shifti_i64(ret, arg1, arg2, 1, 0);
		513	+}
		514	+
		515	+static inline void tcg_gen_sar_i64(int ret, int arg1, int64_t arg2)
		516	+{
		517	+ tcg_gen_helper_1_2(tcg_helper_sar_i64, ret, arg1, arg2);
		518	+}
		519	+
		520	+static inline void tcg_gen_sari_i64(int ret, int arg1, int64_t arg2)
		521	+{
		522	+ tcg_gen_shifti_i64(ret, arg1, arg2, 1, 1);
		523	+}
		524	+
		525	+static inline void tcg_gen_brcond_i64(int cond, TCGArg arg1, TCGArg arg2,
		526	+ int label_index)
		527	+{
		528	+ tcg_gen_op6(INDEX_op_brcond2_i32,
		529	+ arg1, arg1 + 1, arg2, arg2 + 1, cond, label_index);
		530	+}
		531	+
		532	+static inline void tcg_gen_mul_i64(int ret, int arg1, int arg2)
		533	+{
		534	+ int t0, t1;
		535	+
		536	+ t0 = tcg_temp_new(TCG_TYPE_I64);
		537	+ t1 = tcg_temp_new(TCG_TYPE_I32);
		538	+
		539	+ tcg_gen_op4(INDEX_op_mulu2_i32, t0, t0 + 1, arg1, arg2);
		540	+
		541	+ tcg_gen_mul_i32(t1, arg1, arg2 + 1);
		542	+ tcg_gen_add_i32(t0 + 1, t0 + 1, t1);
		543	+ tcg_gen_mul_i32(t1, arg1 + 1, arg2);
		544	+ tcg_gen_add_i32(t0 + 1, t0 + 1, t1);
		545	+
		546	+ tcg_gen_mov_i64(ret, t0);
		547	+}
		548	+
		549	+static inline void tcg_gen_div_i64(int ret, int arg1, int64_t arg2)
		550	+{
		551	+ tcg_gen_helper_1_2(tcg_helper_div_i64, ret, arg1, arg2);
		552	+}
		553	+
		554	+static inline void tcg_gen_rem_i64(int ret, int arg1, int64_t arg2)
		555	+{
		556	+ tcg_gen_helper_1_2(tcg_helper_rem_i64, ret, arg1, arg2);
		557	+}
		558	+
		559	+static inline void tcg_gen_divu_i64(int ret, int arg1, int64_t arg2)
		560	+{
		561	+ tcg_gen_helper_1_2(tcg_helper_divu_i64, ret, arg1, arg2);
		562	+}
		563	+
		564	+static inline void tcg_gen_remu_i64(int ret, int arg1, int64_t arg2)
		565	+{
		566	+ tcg_gen_helper_1_2(tcg_helper_remu_i64, ret, arg1, arg2);
		567	+}
		568	+
		569	+#else
		570	+
		571	+static inline void tcg_gen_mov_i64(int ret, int arg)
		572	+{
		573	+ tcg_gen_op2(INDEX_op_mov_i64, ret, arg);
		574	+}
		575	+
		576	+static inline void tcg_gen_movi_i64(int ret, int64_t arg)
		577	+{
		578	+ tcg_gen_op2(INDEX_op_movi_i64, ret, arg);
		579	+}
		580	+
		581	+static inline void tcg_gen_ld8u_i64(int ret, int arg2, tcg_target_long offset)
		582	+{
		583	+ tcg_gen_op3(INDEX_op_ld8u_i64, ret, arg2, offset);
		584	+}
		585	+
		586	+static inline void tcg_gen_ld8s_i64(int ret, int arg2, tcg_target_long offset)
		587	+{
		588	+ tcg_gen_op3(INDEX_op_ld8s_i64, ret, arg2, offset);
		589	+}
		590	+
		591	+static inline void tcg_gen_ld16u_i64(int ret, int arg2, tcg_target_long offset)
		592	+{
		593	+ tcg_gen_op3(INDEX_op_ld16u_i64, ret, arg2, offset);
		594	+}
		595	+
		596	+static inline void tcg_gen_ld16s_i64(int ret, int arg2, tcg_target_long offset)
		597	+{
		598	+ tcg_gen_op3(INDEX_op_ld16s_i64, ret, arg2, offset);
		599	+}
		600	+
		601	+static inline void tcg_gen_ld32u_i64(int ret, int arg2, tcg_target_long offset)
		602	+{
		603	+ tcg_gen_op3(INDEX_op_ld32u_i64, ret, arg2, offset);
		604	+}
		605	+
		606	+static inline void tcg_gen_ld32s_i64(int ret, int arg2, tcg_target_long offset)
		607	+{
		608	+ tcg_gen_op3(INDEX_op_ld32s_i64, ret, arg2, offset);
		609	+}
		610	+
		611	+static inline void tcg_gen_ld_i64(int ret, int arg2, tcg_target_long offset)
		612	+{
		613	+ tcg_gen_op3(INDEX_op_ld_i64, ret, arg2, offset);
		614	+}
		615	+
		616	+static inline void tcg_gen_st8_i64(int arg1, int arg2, tcg_target_long offset)
		617	+{
		618	+ tcg_gen_op3(INDEX_op_st8_i64, arg1, arg2, offset);
		619	+}
		620	+
		621	+static inline void tcg_gen_st16_i64(int arg1, int arg2, tcg_target_long offset)
		622	+{
		623	+ tcg_gen_op3(INDEX_op_st16_i64, arg1, arg2, offset);
		624	+}
		625	+
		626	+static inline void tcg_gen_st32_i64(int arg1, int arg2, tcg_target_long offset)
		627	+{
		628	+ tcg_gen_op3(INDEX_op_st32_i64, arg1, arg2, offset);
		629	+}
		630	+
		631	+static inline void tcg_gen_st_i64(int arg1, int arg2, tcg_target_long offset)
		632	+{
		633	+ tcg_gen_op3(INDEX_op_st_i64, arg1, arg2, offset);
		634	+}
		635	+
		636	+static inline void tcg_gen_add_i64(int ret, int arg1, int arg2)
		637	+{
		638	+ tcg_gen_op3(INDEX_op_add_i64, ret, arg1, arg2);
		639	+}
		640	+
		641	+static inline void tcg_gen_addi_i64(int ret, int arg1, int64_t arg2)
		642	+{
		643	+ tcg_gen_add_i64(ret, arg1, tcg_const_i64(arg2));
		644	+}
		645	+
		646	+static inline void tcg_gen_sub_i64(int ret, int arg1, int arg2)
		647	+{
		648	+ tcg_gen_op3(INDEX_op_sub_i64, ret, arg1, arg2);
		649	+}
		650	+
		651	+static inline void tcg_gen_subi_i64(int ret, int arg1, int64_t arg2)
		652	+{
		653	+ tcg_gen_sub_i64(ret, arg1, tcg_const_i64(arg2));
		654	+}
		655	+
		656	+static inline void tcg_gen_and_i64(int ret, int arg1, int arg2)
		657	+{
		658	+ tcg_gen_op3(INDEX_op_and_i64, ret, arg1, arg2);
		659	+}
		660	+
		661	+static inline void tcg_gen_andi_i64(int ret, int arg1, int64_t arg2)
		662	+{
		663	+ tcg_gen_and_i64(ret, arg1, tcg_const_i64(arg2));
		664	+}
		665	+
		666	+static inline void tcg_gen_or_i64(int ret, int arg1, int arg2)
		667	+{
		668	+ tcg_gen_op3(INDEX_op_or_i64, ret, arg1, arg2);
		669	+}
		670	+
		671	+static inline void tcg_gen_ori_i64(int ret, int arg1, int64_t arg2)
		672	+{
		673	+ tcg_gen_or_i64(ret, arg1, tcg_const_i64(arg2));
		674	+}
		675	+
		676	+static inline void tcg_gen_xor_i64(int ret, int arg1, int arg2)
		677	+{
		678	+ tcg_gen_op3(INDEX_op_xor_i64, ret, arg1, arg2);
		679	+}
		680	+
		681	+static inline void tcg_gen_xori_i64(int ret, int arg1, int64_t arg2)
		682	+{
		683	+ tcg_gen_xor_i64(ret, arg1, tcg_const_i64(arg2));
		684	+}
		685	+
		686	+static inline void tcg_gen_shl_i64(int ret, int arg1, int arg2)
		687	+{
		688	+ tcg_gen_op3(INDEX_op_shl_i64, ret, arg1, arg2);
		689	+}
		690	+
		691	+static inline void tcg_gen_shli_i64(int ret, int arg1, int64_t arg2)
		692	+{
		693	+ tcg_gen_shl_i64(ret, arg1, tcg_const_i64(arg2));
		694	+}
		695	+
		696	+static inline void tcg_gen_shr_i64(int ret, int arg1, int arg2)
		697	+{
		698	+ tcg_gen_op3(INDEX_op_shr_i64, ret, arg1, arg2);
		699	+}
		700	+
		701	+static inline void tcg_gen_shri_i64(int ret, int arg1, int64_t arg2)
		702	+{
		703	+ tcg_gen_shr_i64(ret, arg1, tcg_const_i64(arg2));
		704	+}
		705	+
		706	+static inline void tcg_gen_sar_i64(int ret, int arg1, int arg2)
		707	+{
		708	+ tcg_gen_op3(INDEX_op_sar_i64, ret, arg1, arg2);
		709	+}
		710	+
		711	+static inline void tcg_gen_sari_i64(int ret, int arg1, int64_t arg2)
		712	+{
		713	+ tcg_gen_sar_i64(ret, arg1, tcg_const_i64(arg2));
		714	+}
		715	+
		716	+static inline void tcg_gen_brcond_i64(int cond, TCGArg arg1, TCGArg arg2,
		717	+ int label_index)
		718	+{
		719	+ tcg_gen_op4(INDEX_op_brcond_i64, arg1, arg2, cond, label_index);
		720	+}
		721	+
		722	+static inline void tcg_gen_mul_i64(int ret, int arg1, int arg2)
		723	+{
		724	+ tcg_gen_op3(INDEX_op_mul_i64, ret, arg1, arg2);
		725	+}
		726	+
		727	+#ifdef TCG_TARGET_HAS_div_i64
		728	+static inline void tcg_gen_div_i64(int ret, int arg1, int arg2)
		729	+{
		730	+ tcg_gen_op3(INDEX_op_div_i64, ret, arg1, arg2);
		731	+}
		732	+
		733	+static inline void tcg_gen_rem_i64(int ret, int arg1, int arg2)
		734	+{
		735	+ tcg_gen_op3(INDEX_op_rem_i64, ret, arg1, arg2);
		736	+}
		737	+
		738	+static inline void tcg_gen_divu_i64(int ret, int arg1, int arg2)
		739	+{
		740	+ tcg_gen_op3(INDEX_op_divu_i64, ret, arg1, arg2);
		741	+}
		742	+
		743	+static inline void tcg_gen_remu_i64(int ret, int arg1, int arg2)
		744	+{
		745	+ tcg_gen_op3(INDEX_op_remu_i64, ret, arg1, arg2);
		746	+}
		747	+#else
		748	+static inline void tcg_gen_div_i64(int ret, int arg1, int arg2)
		749	+{
		750	+ int t0;
		751	+ t0 = tcg_temp_new(TCG_TYPE_I64);
		752	+ tcg_gen_sari_i64(t0, arg1, 63);
		753	+ tcg_gen_op5(INDEX_op_div2_i64, ret, t0, arg1, t0, arg2);
		754	+}
		755	+
		756	+static inline void tcg_gen_rem_i64(int ret, int arg1, int arg2)
		757	+{
		758	+ int t0;
		759	+ t0 = tcg_temp_new(TCG_TYPE_I64);
		760	+ tcg_gen_sari_i64(t0, arg1, 63);
		761	+ tcg_gen_op5(INDEX_op_div2_i64, t0, ret, arg1, t0, arg2);
		762	+}
		763	+
		764	+static inline void tcg_gen_divu_i64(int ret, int arg1, int arg2)
		765	+{
		766	+ int t0;
		767	+ t0 = tcg_temp_new(TCG_TYPE_I64);
		768	+ tcg_gen_movi_i64(t0, 0);
		769	+ tcg_gen_op5(INDEX_op_divu2_i64, ret, t0, arg1, t0, arg2);
		770	+}
		771	+
		772	+static inline void tcg_gen_remu_i64(int ret, int arg1, int arg2)
		773	+{
		774	+ int t0;
		775	+ t0 = tcg_temp_new(TCG_TYPE_I64);
		776	+ tcg_gen_movi_i64(t0, 0);
		777	+ tcg_gen_op5(INDEX_op_divu2_i64, t0, ret, arg1, t0, arg2);
		778	+}
		779	+#endif
		780	+
		781	+#endif
		782	+
		783	+/***************************************/
		784	+/* optional operations */
		785	+
		786	+static inline void tcg_gen_ext8s_i32(int ret, int arg)
		787	+{
		788	+#ifdef TCG_TARGET_HAS_ext8s_i32
		789	+ tcg_gen_op2(INDEX_op_ext8s_i32, ret, arg);
		790	+#else
		791	+ tcg_gen_shli_i32(ret, arg, 24);
		792	+ tcg_gen_sari_i32(ret, arg, 24);
		793	+#endif
		794	+}
		795	+
		796	+static inline void tcg_gen_ext16s_i32(int ret, int arg)
		797	+{
		798	+#ifdef TCG_TARGET_HAS_ext16s_i32
		799	+ tcg_gen_op2(INDEX_op_ext16s_i32, ret, arg);
		800	+#else
		801	+ tcg_gen_shli_i32(ret, arg, 16);
		802	+ tcg_gen_sari_i32(ret, arg, 16);
		803	+#endif
		804	+}
		805	+
		806	+/* Note: we assume the two high bytes are set to zero */
		807	+static inline void tcg_gen_bswap16_i32(TCGArg ret, TCGArg arg)
		808	+{
		809	+#ifdef TCG_TARGET_HAS_bswap16_i32
		810	+ tcg_gen_op2(INDEX_op_bswap16_i32, ret, arg);
		811	+#else
		812	+ TCGArg t0, t1;
		813	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		814	+ t1 = tcg_temp_new(TCG_TYPE_I32);
		815	+
		816	+ tcg_gen_shri_i32(t0, arg, 8);
		817	+ tcg_gen_andi_i32(t1, arg, 0x000000ff);
		818	+ tcg_gen_shli_i32(t1, t1, 8);
		819	+ tcg_gen_or_i32(ret, t0, t1);
		820	+#endif
		821	+}
		822	+
		823	+static inline void tcg_gen_bswap_i32(TCGArg ret, TCGArg arg)
		824	+{
		825	+#ifdef TCG_TARGET_HAS_bswap_i32
		826	+ tcg_gen_op2(INDEX_op_bswap_i32, ret, arg);
		827	+#else
		828	+ TCGArg t0, t1;
		829	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		830	+ t1 = tcg_temp_new(TCG_TYPE_I32);
		831	+
		832	+ tcg_gen_shli_i32(t0, arg, 24);
		833	+
		834	+ tcg_gen_andi_i32(t1, arg, 0x0000ff00);
		835	+ tcg_gen_shli_i32(t1, t1, 8);
		836	+ tcg_gen_or_i32(t0, t0, t1);
		837	+
		838	+ tcg_gen_shri_i32(t1, arg, 8);
		839	+ tcg_gen_andi_i32(t1, t1, 0x0000ff00);
		840	+ tcg_gen_or_i32(t0, t0, t1);
		841	+
		842	+ tcg_gen_shri_i32(t1, arg, 24);
		843	+ tcg_gen_or_i32(ret, t0, t1);
		844	+#endif
		845	+}
		846	+
		847	+#if TCG_TARGET_REG_BITS == 32
		848	+static inline void tcg_gen_ext8s_i64(int ret, int arg)
		849	+{
		850	+ tcg_gen_ext8s_i32(ret, arg);
		851	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		852	+}
		853	+
		854	+static inline void tcg_gen_ext16s_i64(int ret, int arg)
		855	+{
		856	+ tcg_gen_ext16s_i32(ret, arg);
		857	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		858	+}
		859	+
		860	+static inline void tcg_gen_ext32s_i64(int ret, int arg)
		861	+{
		862	+ tcg_gen_mov_i32(ret, arg);
		863	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		864	+}
		865	+
		866	+static inline void tcg_gen_trunc_i64_i32(int ret, int arg)
		867	+{
		868	+ tcg_gen_mov_i32(ret, arg);
		869	+}
		870	+
		871	+static inline void tcg_gen_extu_i32_i64(int ret, int arg)
		872	+{
		873	+ tcg_gen_mov_i32(ret, arg);
		874	+ tcg_gen_movi_i32(ret + 1, 0);
		875	+}
		876	+
		877	+static inline void tcg_gen_ext_i32_i64(int ret, int arg)
		878	+{
		879	+ tcg_gen_mov_i32(ret, arg);
		880	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		881	+}
		882	+
		883	+static inline void tcg_gen_bswap_i64(int ret, int arg)
		884	+{
		885	+ int t0, t1;
		886	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		887	+ t1 = tcg_temp_new(TCG_TYPE_I32);
		888	+
		889	+ tcg_gen_bswap_i32(t0, arg);
		890	+ tcg_gen_bswap_i32(t1, arg + 1);
		891	+ tcg_gen_mov_i32(ret, t1);
		892	+ tcg_gen_mov_i32(ret + 1, t0);
		893	+}
		894	+#else
		895	+
		896	+static inline void tcg_gen_ext8s_i64(int ret, int arg)
		897	+{
		898	+#ifdef TCG_TARGET_HAS_ext8s_i64
		899	+ tcg_gen_op2(INDEX_op_ext8s_i64, ret, arg);
		900	+#else
		901	+ tcg_gen_shli_i64(ret, arg, 56);
		902	+ tcg_gen_sari_i64(ret, arg, 56);
		903	+#endif
		904	+}
		905	+
		906	+static inline void tcg_gen_ext16s_i64(int ret, int arg)
		907	+{
		908	+#ifdef TCG_TARGET_HAS_ext16s_i64
		909	+ tcg_gen_op2(INDEX_op_ext16s_i64, ret, arg);
		910	+#else
		911	+ tcg_gen_shli_i64(ret, arg, 48);
		912	+ tcg_gen_sari_i64(ret, arg, 48);
		913	+#endif
		914	+}
		915	+
		916	+static inline void tcg_gen_ext32s_i64(int ret, int arg)
		917	+{
		918	+#ifdef TCG_TARGET_HAS_ext32s_i64
		919	+ tcg_gen_op2(INDEX_op_ext32s_i64, ret, arg);
		920	+#else
		921	+ tcg_gen_shli_i64(ret, arg, 32);
		922	+ tcg_gen_sari_i64(ret, arg, 32);
		923	+#endif
		924	+}
		925	+
		926	+/* Note: we assume the target supports move between 32 and 64 bit
		927	+ registers */
		928	+static inline void tcg_gen_trunc_i64_i32(int ret, int arg)
		929	+{
		930	+ tcg_gen_mov_i32(ret, arg);
		931	+}
		932	+
		933	+/* Note: we assume the target supports move between 32 and 64 bit
		934	+ registers */
		935	+static inline void tcg_gen_extu_i32_i64(int ret, int arg)
		936	+{
		937	+ tcg_gen_andi_i64(ret, arg, 0xffffffff);
		938	+}
		939	+
		940	+/* Note: we assume the target supports move between 32 and 64 bit
		941	+ registers */
		942	+static inline void tcg_gen_ext_i32_i64(int ret, int arg)
		943	+{
		944	+ tcg_gen_ext32s_i64(ret, arg);
		945	+}
		946	+
		947	+static inline void tcg_gen_bswap_i64(TCGArg ret, TCGArg arg)
		948	+{
		949	+#ifdef TCG_TARGET_HAS_bswap_i64
		950	+ tcg_gen_op2(INDEX_op_bswap_i64, ret, arg);
		951	+#else
		952	+ TCGArg t0, t1;
		953	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		954	+ t1 = tcg_temp_new(TCG_TYPE_I32);
		955	+
		956	+ tcg_gen_shli_i64(t0, arg, 56);
		957	+
		958	+ tcg_gen_andi_i64(t1, arg, 0x0000ff00);
		959	+ tcg_gen_shli_i64(t1, t1, 40);
		960	+ tcg_gen_or_i64(t0, t0, t1);
		961	+
		962	+ tcg_gen_andi_i64(t1, arg, 0x00ff0000);
		963	+ tcg_gen_shli_i64(t1, t1, 24);
		964	+ tcg_gen_or_i64(t0, t0, t1);
		965	+
		966	+ tcg_gen_andi_i64(t1, arg, 0xff000000);
		967	+ tcg_gen_shli_i64(t1, t1, 8);
		968	+ tcg_gen_or_i64(t0, t0, t1);
		969	+
		970	+ tcg_gen_shri_i64(t1, arg, 8);
		971	+ tcg_gen_andi_i64(t1, t1, 0xff000000);
		972	+ tcg_gen_or_i64(t0, t0, t1);
		973	+
		974	+ tcg_gen_shri_i64(t1, arg, 24);
		975	+ tcg_gen_andi_i64(t1, t1, 0x00ff0000);
		976	+ tcg_gen_or_i64(t0, t0, t1);
		977	+
		978	+ tcg_gen_shri_i64(t1, arg, 40);
		979	+ tcg_gen_andi_i64(t1, t1, 0x0000ff00);
		980	+ tcg_gen_or_i64(t0, t0, t1);
		981	+
		982	+ tcg_gen_shri_i64(t1, arg, 56);
		983	+ tcg_gen_or_i64(ret, t0, t1);
		984	+#endif
		985	+}
		986	+
		987	+#endif
		988	+
		989	+/***************************************/
		990	+static inline void tcg_gen_macro_2(int ret0, int ret1, int macro_id)
		991	+{
		992	+ tcg_gen_op3(INDEX_op_macro_2, ret0, ret1, macro_id);
		993	+}
		994	+
		995	+/***************************************/
		996	+/* QEMU specific operations. Their type depend on the QEMU CPU
		997	+ type. */
		998	+#ifndef TARGET_LONG_BITS
		999	+#error must include QEMU headers
		1000	+#endif
		1001	+
		1002	+static inline void tcg_gen_exit_tb(tcg_target_long val)
		1003	+{
		1004	+ tcg_gen_op1(INDEX_op_exit_tb, val);
		1005	+}
		1006	+
		1007	+static inline void tcg_gen_goto_tb(int idx)
		1008	+{
		1009	+ tcg_gen_op1(INDEX_op_goto_tb, idx);
		1010	+}
		1011	+
		1012	+#if TCG_TARGET_REG_BITS == 32
		1013	+static inline void tcg_gen_qemu_ld8u(int ret, int addr, int mem_index)
		1014	+{
		1015	+#if TARGET_LONG_BITS == 32
		1016	+ tcg_gen_op3(INDEX_op_qemu_ld8u, ret, addr, mem_index);
		1017	+#else
		1018	+ tcg_gen_op4(INDEX_op_qemu_ld8u, ret, addr, addr + 1, mem_index);
		1019	+ tcg_gen_movi_i32(ret + 1, 0);
		1020	+#endif
		1021	+}
		1022	+
		1023	+static inline void tcg_gen_qemu_ld8s(int ret, int addr, int mem_index)
		1024	+{
		1025	+#if TARGET_LONG_BITS == 32
		1026	+ tcg_gen_op3(INDEX_op_qemu_ld8s, ret, addr, mem_index);
		1027	+#else
		1028	+ tcg_gen_op4(INDEX_op_qemu_ld8s, ret, addr, addr + 1, mem_index);
		1029	+ tcg_gen_ext8s_i32(ret + 1, ret);
		1030	+#endif
		1031	+}
		1032	+
		1033	+static inline void tcg_gen_qemu_ld16u(int ret, int addr, int mem_index)
		1034	+{
		1035	+#if TARGET_LONG_BITS == 32
		1036	+ tcg_gen_op3(INDEX_op_qemu_ld16u, ret, addr, mem_index);
		1037	+#else
		1038	+ tcg_gen_op4(INDEX_op_qemu_ld16u, ret, addr, addr + 1, mem_index);
		1039	+ tcg_gen_movi_i32(ret + 1, 0);
		1040	+#endif
		1041	+}
		1042	+
		1043	+static inline void tcg_gen_qemu_ld16s(int ret, int addr, int mem_index)
		1044	+{
		1045	+#if TARGET_LONG_BITS == 32
		1046	+ tcg_gen_op3(INDEX_op_qemu_ld16s, ret, addr, mem_index);
		1047	+#else
		1048	+ tcg_gen_op4(INDEX_op_qemu_ld16s, ret, addr, addr + 1, mem_index);
		1049	+ tcg_gen_ext16s_i32(ret + 1, ret);
		1050	+#endif
		1051	+}
		1052	+
		1053	+static inline void tcg_gen_qemu_ld32u(int ret, int addr, int mem_index)
		1054	+{
		1055	+#if TARGET_LONG_BITS == 32
		1056	+ tcg_gen_op3(INDEX_op_qemu_ld32u, ret, addr, mem_index);
		1057	+#else
		1058	+ tcg_gen_op4(INDEX_op_qemu_ld32u, ret, addr, addr + 1, mem_index);
		1059	+ tcg_gen_movi_i32(ret + 1, 0);
		1060	+#endif
		1061	+}
		1062	+
		1063	+static inline void tcg_gen_qemu_ld32s(int ret, int addr, int mem_index)
		1064	+{
		1065	+#if TARGET_LONG_BITS == 32
		1066	+ tcg_gen_op3(INDEX_op_qemu_ld32u, ret, addr, mem_index);
		1067	+#else
		1068	+ tcg_gen_op4(INDEX_op_qemu_ld32u, ret, addr, addr + 1, mem_index);
		1069	+ tcg_gen_sari_i32(ret + 1, ret, 31);
		1070	+#endif
		1071	+}
		1072	+
		1073	+static inline void tcg_gen_qemu_ld64(int ret, int addr, int mem_index)
		1074	+{
		1075	+#if TARGET_LONG_BITS == 32
		1076	+ tcg_gen_op4(INDEX_op_qemu_ld64, ret, ret + 1, addr, mem_index);
		1077	+#else
		1078	+ tcg_gen_op5(INDEX_op_qemu_ld64, ret, ret + 1, addr, addr + 1, mem_index);
		1079	+#endif
		1080	+}
		1081	+
		1082	+static inline void tcg_gen_qemu_st8(int arg, int addr, int mem_index)
		1083	+{
		1084	+#if TARGET_LONG_BITS == 32
		1085	+ tcg_gen_op3(INDEX_op_qemu_st8, arg, addr, mem_index);
		1086	+#else
		1087	+ tcg_gen_op4(INDEX_op_qemu_st8, arg, addr, addr + 1, mem_index);
		1088	+#endif
		1089	+}
		1090	+
		1091	+static inline void tcg_gen_qemu_st16(int arg, int addr, int mem_index)
		1092	+{
		1093	+#if TARGET_LONG_BITS == 32
		1094	+ tcg_gen_op3(INDEX_op_qemu_st16, arg, addr, mem_index);
		1095	+#else
		1096	+ tcg_gen_op4(INDEX_op_qemu_st16, arg, addr, addr + 1, mem_index);
		1097	+#endif
		1098	+}
		1099	+
		1100	+static inline void tcg_gen_qemu_st32(int arg, int addr, int mem_index)
		1101	+{
		1102	+#if TARGET_LONG_BITS == 32
		1103	+ tcg_gen_op3(INDEX_op_qemu_st32, arg, addr, mem_index);
		1104	+#else
		1105	+ tcg_gen_op4(INDEX_op_qemu_st32, arg, addr, addr + 1, mem_index);
		1106	+#endif
		1107	+}
		1108	+
		1109	+static inline void tcg_gen_qemu_st64(int arg, int addr, int mem_index)
		1110	+{
		1111	+#if TARGET_LONG_BITS == 32
		1112	+ tcg_gen_op4(INDEX_op_qemu_st64, arg, arg + 1, addr, mem_index);
		1113	+#else
		1114	+ tcg_gen_op5(INDEX_op_qemu_st64, arg, arg + 1, addr, addr + 1, mem_index);
		1115	+#endif
		1116	+}
		1117	+
		1118	+#else /* TCG_TARGET_REG_BITS == 32 */
		1119	+
		1120	+static inline void tcg_gen_qemu_ld8u(int ret, int addr, int mem_index)
		1121	+{
		1122	+ tcg_gen_op3(INDEX_op_qemu_ld8u, ret, addr, mem_index);
		1123	+}
		1124	+
		1125	+static inline void tcg_gen_qemu_ld8s(int ret, int addr, int mem_index)
		1126	+{
		1127	+ tcg_gen_op3(INDEX_op_qemu_ld8s, ret, addr, mem_index);
		1128	+}
		1129	+
		1130	+static inline void tcg_gen_qemu_ld16u(int ret, int addr, int mem_index)
		1131	+{
		1132	+ tcg_gen_op3(INDEX_op_qemu_ld16u, ret, addr, mem_index);
		1133	+}
		1134	+
		1135	+static inline void tcg_gen_qemu_ld16s(int ret, int addr, int mem_index)
		1136	+{
		1137	+ tcg_gen_op3(INDEX_op_qemu_ld16s, ret, addr, mem_index);
		1138	+}
		1139	+
		1140	+static inline void tcg_gen_qemu_ld32u(int ret, int addr, int mem_index)
		1141	+{
		1142	+ tcg_gen_op3(INDEX_op_qemu_ld32u, ret, addr, mem_index);
		1143	+}
		1144	+
		1145	+static inline void tcg_gen_qemu_ld32s(int ret, int addr, int mem_index)
		1146	+{
		1147	+ tcg_gen_op3(INDEX_op_qemu_ld32s, ret, addr, mem_index);
		1148	+}
		1149	+
		1150	+static inline void tcg_gen_qemu_ld64(int ret, int addr, int mem_index)
		1151	+{
		1152	+ tcg_gen_op3(INDEX_op_qemu_ld64, ret, addr, mem_index);
		1153	+}
		1154	+
		1155	+static inline void tcg_gen_qemu_st8(int arg, int addr, int mem_index)
		1156	+{
		1157	+ tcg_gen_op3(INDEX_op_qemu_st8, arg, addr, mem_index);
		1158	+}
		1159	+
		1160	+static inline void tcg_gen_qemu_st16(int arg, int addr, int mem_index)
		1161	+{
		1162	+ tcg_gen_op3(INDEX_op_qemu_st16, arg, addr, mem_index);
		1163	+}
		1164	+
		1165	+static inline void tcg_gen_qemu_st32(int arg, int addr, int mem_index)
		1166	+{
		1167	+ tcg_gen_op3(INDEX_op_qemu_st32, arg, addr, mem_index);
		1168	+}
		1169	+
		1170	+static inline void tcg_gen_qemu_st64(int arg, int addr, int mem_index)
		1171	+{
		1172	+ tcg_gen_op3(INDEX_op_qemu_st64, arg, addr, mem_index);
		1173	+}
		1174	+
		1175	+#endif /* TCG_TARGET_REG_BITS != 32 */

tcg/tcg-opc.h 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#include "dyngen-opc.h"
		25	+
		26	+#ifndef DEF2
		27	+#define DEF2(name, oargs, iargs, cargs, flags) DEF(name, oargs + iargs + cargs, 0)
		28	+#endif
		29	+
		30	+/* predefined ops */
		31	+DEF2(end, 0, 0, 0, 0) /* must be kept first */
		32	+DEF2(nop, 0, 0, 0, 0)
		33	+DEF2(nop1, 0, 0, 1, 0)
		34	+DEF2(nop2, 0, 0, 2, 0)
		35	+DEF2(nop3, 0, 0, 3, 0)
		36	+DEF2(nopn, 0, 0, 1, 0) /* variable number of parameters */
		37	+/* macro handling */
		38	+DEF2(macro_2, 2, 0, 1, 0)
		39	+DEF2(macro_start, 0, 0, 2, 0)
		40	+DEF2(macro_end, 0, 0, 2, 0)
		41	+DEF2(macro_goto, 0, 0, 3, 0)
		42	+
		43	+DEF2(set_label, 0, 0, 1, 0)
		44	+DEF2(call, 0, 1, 2, 0) /* variable number of parameters */
		45	+DEF2(jmp, 0, 1, 0, TCG_OPF_BB_END)
		46	+DEF2(br, 0, 0, 1, TCG_OPF_BB_END)
		47	+
		48	+DEF2(mov_i32, 1, 1, 0, 0)
		49	+DEF2(movi_i32, 1, 0, 1, 0)
		50	+/* load/store */
		51	+DEF2(ld8u_i32, 1, 1, 1, 0)
		52	+DEF2(ld8s_i32, 1, 1, 1, 0)
		53	+DEF2(ld16u_i32, 1, 1, 1, 0)
		54	+DEF2(ld16s_i32, 1, 1, 1, 0)
		55	+DEF2(ld_i32, 1, 1, 1, 0)
		56	+DEF2(st8_i32, 0, 2, 1, 0)
		57	+DEF2(st16_i32, 0, 2, 1, 0)
		58	+DEF2(st_i32, 0, 2, 1, 0)
		59	+/* arith */
		60	+DEF2(add_i32, 1, 2, 0, 0)
		61	+DEF2(sub_i32, 1, 2, 0, 0)
		62	+DEF2(mul_i32, 1, 2, 0, 0)
		63	+#ifdef TCG_TARGET_HAS_div_i32
		64	+DEF2(div_i32, 1, 2, 0, 0)
		65	+DEF2(divu_i32, 1, 2, 0, 0)
		66	+DEF2(rem_i32, 1, 2, 0, 0)
		67	+DEF2(remu_i32, 1, 2, 0, 0)
		68	+#else
		69	+DEF2(div2_i32, 2, 3, 0, 0)
		70	+DEF2(divu2_i32, 2, 3, 0, 0)
		71	+#endif
		72	+DEF2(and_i32, 1, 2, 0, 0)
		73	+DEF2(or_i32, 1, 2, 0, 0)
		74	+DEF2(xor_i32, 1, 2, 0, 0)
		75	+/* shifts */
		76	+DEF2(shl_i32, 1, 2, 0, 0)
		77	+DEF2(shr_i32, 1, 2, 0, 0)
		78	+DEF2(sar_i32, 1, 2, 0, 0)
		79	+
		80	+DEF2(brcond_i32, 0, 2, 2, TCG_OPF_BB_END)
		81	+#if TCG_TARGET_REG_BITS == 32
		82	+DEF2(add2_i32, 2, 4, 0, 0)
		83	+DEF2(sub2_i32, 2, 4, 0, 0)
		84	+DEF2(brcond2_i32, 0, 4, 2, TCG_OPF_BB_END)
		85	+DEF2(mulu2_i32, 2, 2, 0, 0)
		86	+#endif
		87	+#ifdef TCG_TARGET_HAS_ext8s_i32
		88	+DEF2(ext8s_i32, 1, 1, 0, 0)
		89	+#endif
		90	+#ifdef TCG_TARGET_HAS_ext16s_i32
		91	+DEF2(ext16s_i32, 1, 1, 0, 0)
		92	+#endif
		93	+#ifdef TCG_TARGET_HAS_bswap_i32
		94	+DEF2(bswap_i32, 1, 1, 0, 0)
		95	+#endif
		96	+
		97	+#if TCG_TARGET_REG_BITS == 64
		98	+DEF2(mov_i64, 1, 1, 0, 0)
		99	+DEF2(movi_i64, 1, 0, 1, 0)
		100	+/* load/store */
		101	+DEF2(ld8u_i64, 1, 1, 1, 0)
		102	+DEF2(ld8s_i64, 1, 1, 1, 0)
		103	+DEF2(ld16u_i64, 1, 1, 1, 0)
		104	+DEF2(ld16s_i64, 1, 1, 1, 0)
		105	+DEF2(ld32u_i64, 1, 1, 1, 0)
		106	+DEF2(ld32s_i64, 1, 1, 1, 0)
		107	+DEF2(ld_i64, 1, 1, 1, 0)
		108	+DEF2(st8_i64, 0, 2, 1, 0)
		109	+DEF2(st16_i64, 0, 2, 1, 0)
		110	+DEF2(st32_i64, 0, 2, 1, 0)
		111	+DEF2(st_i64, 0, 2, 1, 0)
		112	+/* arith */
		113	+DEF2(add_i64, 1, 2, 0, 0)
		114	+DEF2(sub_i64, 1, 2, 0, 0)
		115	+DEF2(mul_i64, 1, 2, 0, 0)
		116	+#ifdef TCG_TARGET_HAS_div_i64
		117	+DEF2(div_i64, 1, 2, 0, 0)
		118	+DEF2(divu_i64, 1, 2, 0, 0)
		119	+DEF2(rem_i64, 1, 2, 0, 0)
		120	+DEF2(remu_i64, 1, 2, 0, 0)
		121	+#else
		122	+DEF2(div2_i64, 2, 3, 0, 0)
		123	+DEF2(divu2_i64, 2, 3, 0, 0)
		124	+#endif
		125	+DEF2(and_i64, 1, 2, 0, 0)
		126	+DEF2(or_i64, 1, 2, 0, 0)
		127	+DEF2(xor_i64, 1, 2, 0, 0)
		128	+/* shifts */
		129	+DEF2(shl_i64, 1, 2, 0, 0)
		130	+DEF2(shr_i64, 1, 2, 0, 0)
		131	+DEF2(sar_i64, 1, 2, 0, 0)
		132	+
		133	+DEF2(brcond_i64, 0, 2, 2, TCG_OPF_BB_END)
		134	+#ifdef TCG_TARGET_HAS_ext8s_i64
		135	+DEF2(ext8s_i64, 1, 1, 0, 0)
		136	+#endif
		137	+#ifdef TCG_TARGET_HAS_ext16s_i64
		138	+DEF2(ext16s_i64, 1, 1, 0, 0)
		139	+#endif
		140	+#ifdef TCG_TARGET_HAS_ext32s_i64
		141	+DEF2(ext32s_i64, 1, 1, 0, 0)
		142	+#endif
		143	+#ifdef TCG_TARGET_HAS_bswap_i64
		144	+DEF2(bswap_i64, 1, 1, 0, 0)
		145	+#endif
		146	+#endif
		147	+
		148	+/* QEMU specific */
		149	+DEF2(exit_tb, 0, 0, 1, TCG_OPF_BB_END)
		150	+DEF2(goto_tb, 0, 0, 1, TCG_OPF_BB_END)
		151	+/* Note: even if TARGET_LONG_BITS is not defined, the INDEX_op
		152	+ constants must be defined */
		153	+#if TCG_TARGET_REG_BITS == 32
		154	+#if TARGET_LONG_BITS == 32
		155	+DEF2(qemu_ld8u, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		156	+#else
		157	+DEF2(qemu_ld8u, 1, 2, 1, TCG_OPF_CALL_CLOBBER)
		158	+#endif
		159	+#if TARGET_LONG_BITS == 32
		160	+DEF2(qemu_ld8s, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		161	+#else
		162	+DEF2(qemu_ld8s, 1, 2, 1, TCG_OPF_CALL_CLOBBER)
		163	+#endif
		164	+#if TARGET_LONG_BITS == 32
		165	+DEF2(qemu_ld16u, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		166	+#else
		167	+DEF2(qemu_ld16u, 1, 2, 1, TCG_OPF_CALL_CLOBBER)
		168	+#endif
		169	+#if TARGET_LONG_BITS == 32
		170	+DEF2(qemu_ld16s, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		171	+#else
		172	+DEF2(qemu_ld16s, 1, 2, 1, TCG_OPF_CALL_CLOBBER)
		173	+#endif
		174	+#if TARGET_LONG_BITS == 32
		175	+DEF2(qemu_ld32u, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		176	+#else
		177	+DEF2(qemu_ld32u, 1, 2, 1, TCG_OPF_CALL_CLOBBER)
		178	+#endif
		179	+#if TARGET_LONG_BITS == 32
		180	+DEF2(qemu_ld32s, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		181	+#else
		182	+DEF2(qemu_ld32s, 1, 2, 1, TCG_OPF_CALL_CLOBBER)
		183	+#endif
		184	+#if TARGET_LONG_BITS == 32
		185	+DEF2(qemu_ld64, 2, 1, 1, TCG_OPF_CALL_CLOBBER)
		186	+#else
		187	+DEF2(qemu_ld64, 2, 2, 1, TCG_OPF_CALL_CLOBBER)
		188	+#endif
		189	+
		190	+#if TARGET_LONG_BITS == 32
		191	+DEF2(qemu_st8, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		192	+#else
		193	+DEF2(qemu_st8, 0, 3, 1, TCG_OPF_CALL_CLOBBER)
		194	+#endif
		195	+#if TARGET_LONG_BITS == 32
		196	+DEF2(qemu_st16, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		197	+#else
		198	+DEF2(qemu_st16, 0, 3, 1, TCG_OPF_CALL_CLOBBER)
		199	+#endif
		200	+#if TARGET_LONG_BITS == 32
		201	+DEF2(qemu_st32, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		202	+#else
		203	+DEF2(qemu_st32, 0, 3, 1, TCG_OPF_CALL_CLOBBER)
		204	+#endif
		205	+#if TARGET_LONG_BITS == 32
		206	+DEF2(qemu_st64, 0, 3, 1, TCG_OPF_CALL_CLOBBER)
		207	+#else
		208	+DEF2(qemu_st64, 0, 4, 1, TCG_OPF_CALL_CLOBBER)
		209	+#endif
		210	+
		211	+#else /* TCG_TARGET_REG_BITS == 32 */
		212	+
		213	+DEF2(qemu_ld8u, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		214	+DEF2(qemu_ld8s, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		215	+DEF2(qemu_ld16u, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		216	+DEF2(qemu_ld16s, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		217	+DEF2(qemu_ld32u, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		218	+DEF2(qemu_ld32s, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		219	+DEF2(qemu_ld64, 1, 1, 1, TCG_OPF_CALL_CLOBBER)
		220	+
		221	+DEF2(qemu_st8, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		222	+DEF2(qemu_st16, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		223	+DEF2(qemu_st32, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		224	+DEF2(qemu_st64, 0, 2, 1, TCG_OPF_CALL_CLOBBER)
		225	+
		226	+#endif /* TCG_TARGET_REG_BITS != 32 */
		227	+
		228	+#undef DEF2

tcg/tcg-runtime.c 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#include <stdarg.h>
		25	+#include <stdlib.h>
		26	+#include <stdio.h>
		27	+#include <string.h>
		28	+#include <inttypes.h>
		29	+
		30	+#include "config.h"
		31	+#include "osdep.h"
		32	+#include "tcg.h"
		33	+
		34	+int64_t tcg_helper_shl_i64(int64_t arg1, int64_t arg2)
		35	+{
		36	+ return arg1 << arg2;
		37	+}
		38	+
		39	+int64_t tcg_helper_shr_i64(int64_t arg1, int64_t arg2)
		40	+{
		41	+ return (uint64_t)arg1 >> arg2;
		42	+}
		43	+
		44	+int64_t tcg_helper_sar_i64(int64_t arg1, int64_t arg2)
		45	+{
		46	+ return arg1 >> arg2;
		47	+}
		48	+
		49	+int64_t tcg_helper_div_i64(int64_t arg1, int64_t arg2)
		50	+{
		51	+ return arg1 / arg2;
		52	+}
		53	+
		54	+int64_t tcg_helper_rem_i64(int64_t arg1, int64_t arg2)
		55	+{
		56	+ return arg1 / arg2;
		57	+}
		58	+
		59	+uint64_t tcg_helper_divu_i64(uint64_t arg1, uint64_t arg2)
		60	+{
		61	+ return arg1 / arg2;
		62	+}
		63	+
		64	+uint64_t tcg_helper_remu_i64(uint64_t arg1, uint64_t arg2)
		65	+{
		66	+ return arg1 / arg2;
		67	+}
		68	+

tcg/tcg.c 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+
		25	+/* define it to suppress various consistency checks (faster) */
		26	+#define NDEBUG
		27	+
		28	+/* define it to use liveness analysis (better code) */
		29	+#define USE_LIVENESS_ANALYSIS
		30	+
		31	+#include <assert.h>
		32	+#include <stdarg.h>
		33	+#include <stdlib.h>
		34	+#include <stdio.h>
		35	+#include <string.h>
		36	+#include <inttypes.h>
		37	+
		38	+#include "config.h"
		39	+#include "osdep.h"
		40	+
		41	+/* Note: the long term plan is to reduce the dependancies on the QEMU
		42	+ CPU definitions. Currently they are used for qemu_ld/st
		43	+ instructions */
		44	+#define NO_CPU_IO_DEFS
		45	+#include "cpu.h"
		46	+#include "exec-all.h"
		47	+
		48	+#include "tcg-op.h"
		49	+#include "elf.h"
		50	+
		51	+
		52	+static void patch_reloc(uint8_t *code_ptr, int type,
		53	+ tcg_target_long value);
		54	+
		55	+TCGOpDef tcg_op_defs[] = {
		56	+#define DEF(s, n, copy_size) { #s, 0, 0, n, n, 0, copy_size },
		57	+#define DEF2(s, iargs, oargs, cargs, flags) { #s, iargs, oargs, cargs, iargs + oargs + cargs, flags, 0 },
		58	+#include "tcg-opc.h"
		59	+#undef DEF
		60	+#undef DEF2
		61	+};
		62	+
		63	+TCGRegSet tcg_target_available_regs[2];
		64	+TCGRegSet tcg_target_call_clobber_regs;
		65	+
		66	+/* XXX: move that inside the context */
		67	+uint16_t *gen_opc_ptr;
		68	+TCGArg *gen_opparam_ptr;
		69	+
		70	+static inline void tcg_out8(TCGContext *s, uint8_t v)
		71	+{
		72	+ *s->code_ptr++ = v;
		73	+}
		74	+
		75	+static inline void tcg_out16(TCGContext *s, uint16_t v)
		76	+{
		77	+ (uint16_t )s->code_ptr = v;
		78	+ s->code_ptr += 2;
		79	+}
		80	+
		81	+static inline void tcg_out32(TCGContext *s, uint32_t v)
		82	+{
		83	+ (uint32_t )s->code_ptr = v;
		84	+ s->code_ptr += 4;
		85	+}
		86	+
		87	+/* label relocation processing */
		88	+
		89	+void tcg_out_reloc(TCGContext s, uint8_t code_ptr, int type,
		90	+ int label_index, long addend)
		91	+{
		92	+ TCGLabel *l;
		93	+ TCGRelocation *r;
		94	+
		95	+ l = &s->labels[label_index];
		96	+ if (l->has_value) {
		97	+ patch_reloc(code_ptr, type, l->u.value + addend);
		98	+ } else {
		99	+ /* add a new relocation entry */
		100	+ r = tcg_malloc(sizeof(TCGRelocation));
		101	+ r->type = type;
		102	+ r->ptr = code_ptr;
		103	+ r->addend = addend;
		104	+ r->next = l->u.first_reloc;
		105	+ l->u.first_reloc = r;
		106	+ }
		107	+}
		108	+
		109	+static void tcg_out_label(TCGContext *s, int label_index,
		110	+ tcg_target_long value)
		111	+{
		112	+ TCGLabel *l;
		113	+ TCGRelocation *r;
		114	+
		115	+ l = &s->labels[label_index];
		116	+ if (l->has_value)
		117	+ tcg_abort();
		118	+ r = l->u.first_reloc;
		119	+ while (r != NULL) {
		120	+ patch_reloc(r->ptr, r->type, value + r->addend);
		121	+ r = r->next;
		122	+ }
		123	+ l->has_value = 1;
		124	+ l->u.value = value;
		125	+}
		126	+
		127	+int gen_new_label(void)
		128	+{
		129	+ TCGContext *s = &tcg_ctx;
		130	+ int idx;
		131	+ TCGLabel *l;
		132	+
		133	+ if (s->nb_labels >= TCG_MAX_LABELS)
		134	+ tcg_abort();
		135	+ idx = s->nb_labels++;
		136	+ l = &s->labels[idx];
		137	+ l->has_value = 0;
		138	+ l->u.first_reloc = NULL;
		139	+ return idx;
		140	+}
		141	+
		142	+#include "tcg-target.c"
		143	+
		144	+/* XXX: factorize */
		145	+static void pstrcpy(char buf, int buf_size, const char str)
		146	+{
		147	+ int c;
		148	+ char *q = buf;
		149	+
		150	+ if (buf_size <= 0)
		151	+ return;
		152	+
		153	+ for(;;) {
		154	+ c = *str++;
		155	+ if (c == 0 \|\| q >= buf + buf_size - 1)
		156	+ break;
		157	+ *q++ = c;
		158	+ }
		159	+ *q = '\0';
		160	+}
		161	+
		162	+#if TCG_TARGET_REG_BITS == 32
		163	+/* strcat and truncate. */
		164	+static char pstrcat(char buf, int buf_size, const char *s)
		165	+{
		166	+ int len;
		167	+ len = strlen(buf);
		168	+ if (len < buf_size)
		169	+ pstrcpy(buf + len, buf_size - len, s);
		170	+ return buf;
		171	+}
		172	+#endif
		173	+
		174	+/* pool based memory allocation */
		175	+void tcg_malloc_internal(TCGContext s, int size)
		176	+{
		177	+ TCGPool *p;
		178	+ int pool_size;
		179	+
		180	+ if (size > TCG_POOL_CHUNK_SIZE) {
		181	+ /* big malloc: insert a new pool (XXX: could optimize) */
		182	+ p = qemu_malloc(sizeof(TCGPool) + size);
		183	+ p->size = size;
		184	+ if (s->pool_current)
		185	+ s->pool_current->next = p;
		186	+ else
		187	+ s->pool_first = p;
		188	+ p->next = s->pool_current;
		189	+ } else {
		190	+ p = s->pool_current;
		191	+ if (!p) {
		192	+ p = s->pool_first;
		193	+ if (!p)
		194	+ goto new_pool;
		195	+ } else {
		196	+ if (!p->next) {
		197	+ new_pool:
		198	+ pool_size = TCG_POOL_CHUNK_SIZE;
		199	+ p = qemu_malloc(sizeof(TCGPool) + pool_size);
		200	+ p->size = pool_size;
		201	+ p->next = NULL;
		202	+ if (s->pool_current)
		203	+ s->pool_current->next = p;
		204	+ else
		205	+ s->pool_first = p;
		206	+ } else {
		207	+ p = p->next;
		208	+ }
		209	+ }
		210	+ }
		211	+ s->pool_current = p;
		212	+ s->pool_cur = p->data + size;
		213	+ s->pool_end = p->data + p->size;
		214	+ return p->data;
		215	+}
		216	+
		217	+void tcg_pool_reset(TCGContext *s)
		218	+{
		219	+ s->pool_cur = s->pool_end = NULL;
		220	+ s->pool_current = NULL;
		221	+}
		222	+
		223	+/* free all the pool */
		224	+void tcg_pool_free(TCGContext *s)
		225	+{
		226	+ TCGPool p, p1;
		227	+
		228	+ for(p = s->pool_first; p != NULL; p = p1) {
		229	+ p1 = p->next;
		230	+ qemu_free(p);
		231	+ }
		232	+ s->pool_first = NULL;
		233	+ s->pool_cur = s->pool_end = NULL;
		234	+}
		235	+
		236	+void tcg_context_init(TCGContext *s)
		237	+{
		238	+ int op, total_args, n;
		239	+ TCGOpDef *def;
		240	+ TCGArgConstraint *args_ct;
		241	+ int *sorted_args;
		242	+
		243	+ memset(s, 0, sizeof(*s));
		244	+ s->temps = s->static_temps;
		245	+ s->nb_globals = 0;
		246	+
		247	+ /* Count total number of arguments and allocate the corresponding
		248	+ space */
		249	+ total_args = 0;
		250	+ for(op = 0; op < NB_OPS; op++) {
		251	+ def = &tcg_op_defs[op];
		252	+ n = def->nb_iargs + def->nb_oargs;
		253	+ total_args += n;
		254	+ }
		255	+
		256	+ args_ct = qemu_malloc(sizeof(TCGArgConstraint) * total_args);
		257	+ sorted_args = qemu_malloc(sizeof(int) * total_args);
		258	+
		259	+ for(op = 0; op < NB_OPS; op++) {
		260	+ def = &tcg_op_defs[op];
		261	+ def->args_ct = args_ct;
		262	+ def->sorted_args = sorted_args;
		263	+ n = def->nb_iargs + def->nb_oargs;
		264	+ sorted_args += n;
		265	+ args_ct += n;
		266	+ }
		267	+
		268	+ tcg_target_init(s);
		269	+}
		270	+
		271	+void tcg_set_frame(TCGContext *s, int reg,
		272	+ tcg_target_long start, tcg_target_long size)
		273	+{
		274	+ s->frame_start = start;
		275	+ s->frame_end = start + size;
		276	+ s->frame_reg = reg;
		277	+}
		278	+
		279	+void tcg_set_macro_func(TCGContext s, TCGMacroFunc func)
		280	+{
		281	+ s->macro_func = func;
		282	+}
		283	+
		284	+void tcg_func_start(TCGContext *s)
		285	+{
		286	+ tcg_pool_reset(s);
		287	+ s->nb_temps = s->nb_globals;
		288	+ s->labels = tcg_malloc(sizeof(TCGLabel) * TCG_MAX_LABELS);
		289	+ s->nb_labels = 0;
		290	+ s->current_frame_offset = s->frame_start;
		291	+
		292	+ gen_opc_ptr = gen_opc_buf;
		293	+ gen_opparam_ptr = gen_opparam_buf;
		294	+}
		295	+
		296	+static inline void tcg_temp_alloc(TCGContext *s, int n)
		297	+{
		298	+ if (n > TCG_MAX_TEMPS)
		299	+ tcg_abort();
		300	+}
		301	+
		302	+int tcg_global_reg_new(TCGType type, int reg, const char *name)
		303	+{
		304	+ TCGContext *s = &tcg_ctx;
		305	+ TCGTemp *ts;
		306	+ int idx;
		307	+
		308	+#if TCG_TARGET_REG_BITS == 32
		309	+ if (type != TCG_TYPE_I32)
		310	+ tcg_abort();
		311	+#endif
		312	+ if (tcg_regset_test_reg(s->reserved_regs, reg))
		313	+ tcg_abort();
		314	+ idx = s->nb_globals;
		315	+ tcg_temp_alloc(s, s->nb_globals + 1);
		316	+ ts = &s->temps[s->nb_globals];
		317	+ ts->base_type = type;
		318	+ ts->type = type;
		319	+ ts->fixed_reg = 1;
		320	+ ts->reg = reg;
		321	+ ts->val_type = TEMP_VAL_REG;
		322	+ ts->name = name;
		323	+ s->nb_globals++;
		324	+ tcg_regset_set_reg(s->reserved_regs, reg);
		325	+ return idx;
		326	+}
		327	+
		328	+int tcg_global_mem_new(TCGType type, int reg, tcg_target_long offset,
		329	+ const char *name)
		330	+{
		331	+ TCGContext *s = &tcg_ctx;
		332	+ TCGTemp *ts;
		333	+ int idx;
		334	+
		335	+ idx = s->nb_globals;
		336	+#if TCG_TARGET_REG_BITS == 32
		337	+ if (type == TCG_TYPE_I64) {
		338	+ char buf[64];
		339	+ tcg_temp_alloc(s, s->nb_globals + 1);
		340	+ ts = &s->temps[s->nb_globals];
		341	+ ts->base_type = type;
		342	+ ts->type = TCG_TYPE_I32;
		343	+ ts->fixed_reg = 0;
		344	+ ts->mem_allocated = 1;
		345	+ ts->mem_reg = reg;
		346	+#ifdef TCG_TARGET_WORDS_BIGENDIAN
		347	+ ts->mem_offset = offset + 4;
		348	+#else
		349	+ ts->mem_offset = offset;
		350	+#endif
		351	+ ts->val_type = TEMP_VAL_MEM;
		352	+ pstrcpy(buf, sizeof(buf), name);
		353	+ pstrcat(buf, sizeof(buf), "_0");
		354	+ ts->name = strdup(buf);
		355	+ ts++;
		356	+
		357	+ ts->base_type = type;
		358	+ ts->type = TCG_TYPE_I32;
		359	+ ts->fixed_reg = 0;
		360	+ ts->mem_allocated = 1;
		361	+ ts->mem_reg = reg;
		362	+#ifdef TCG_TARGET_WORDS_BIGENDIAN
		363	+ ts->mem_offset = offset;
		364	+#else
		365	+ ts->mem_offset = offset + 4;
		366	+#endif
		367	+ ts->val_type = TEMP_VAL_MEM;
		368	+ pstrcpy(buf, sizeof(buf), name);
		369	+ pstrcat(buf, sizeof(buf), "_1");
		370	+ ts->name = strdup(buf);
		371	+
		372	+ s->nb_globals += 2;
		373	+ } else
		374	+#endif
		375	+ {
		376	+ tcg_temp_alloc(s, s->nb_globals + 1);
		377	+ ts = &s->temps[s->nb_globals];
		378	+ ts->base_type = type;
		379	+ ts->type = type;
		380	+ ts->fixed_reg = 0;
		381	+ ts->mem_allocated = 1;
		382	+ ts->mem_reg = reg;
		383	+ ts->mem_offset = offset;
		384	+ ts->val_type = TEMP_VAL_MEM;
		385	+ ts->name = name;
		386	+ s->nb_globals++;
		387	+ }
		388	+ return idx;
		389	+}
		390	+
		391	+int tcg_temp_new(TCGType type)
		392	+{
		393	+ TCGContext *s = &tcg_ctx;
		394	+ TCGTemp *ts;
		395	+ int idx;
		396	+
		397	+ idx = s->nb_temps;
		398	+#if TCG_TARGET_REG_BITS == 32
		399	+ if (type == TCG_TYPE_I64) {
		400	+ tcg_temp_alloc(s, s->nb_temps + 1);
		401	+ ts = &s->temps[s->nb_temps];
		402	+ ts->base_type = type;
		403	+ ts->type = TCG_TYPE_I32;
		404	+ ts->val_type = TEMP_VAL_DEAD;
		405	+ ts->mem_allocated = 0;
		406	+ ts->name = NULL;
		407	+ ts++;
		408	+ ts->base_type = TCG_TYPE_I32;
		409	+ ts->type = TCG_TYPE_I32;
		410	+ ts->val_type = TEMP_VAL_DEAD;
		411	+ ts->mem_allocated = 0;
		412	+ ts->name = NULL;
		413	+ s->nb_temps += 2;
		414	+ } else
		415	+#endif
		416	+ {
		417	+ tcg_temp_alloc(s, s->nb_temps + 1);
		418	+ ts = &s->temps[s->nb_temps];
		419	+ ts->base_type = type;
		420	+ ts->type = type;
		421	+ ts->val_type = TEMP_VAL_DEAD;
		422	+ ts->mem_allocated = 0;
		423	+ ts->name = NULL;
		424	+ s->nb_temps++;
		425	+ }
		426	+ return idx;
		427	+}
		428	+
		429	+int tcg_const_i32(int32_t val)
		430	+{
		431	+ TCGContext *s = &tcg_ctx;
		432	+ TCGTemp *ts;
		433	+ int idx;
		434	+
		435	+ idx = s->nb_temps;
		436	+ tcg_temp_alloc(s, idx + 1);
		437	+ ts = &s->temps[idx];
		438	+ ts->base_type = ts->type = TCG_TYPE_I32;
		439	+ ts->val_type = TEMP_VAL_CONST;
		440	+ ts->name = NULL;
		441	+ ts->val = val;
		442	+ s->nb_temps++;
		443	+ return idx;
		444	+}
		445	+
		446	+int tcg_const_i64(int64_t val)
		447	+{
		448	+ TCGContext *s = &tcg_ctx;
		449	+ TCGTemp *ts;
		450	+ int idx;
		451	+
		452	+ idx = s->nb_temps;
		453	+#if TCG_TARGET_REG_BITS == 32
		454	+ tcg_temp_alloc(s, idx + 2);
		455	+ ts = &s->temps[idx];
		456	+ ts->base_type = TCG_TYPE_I64;
		457	+ ts->type = TCG_TYPE_I32;
		458	+ ts->val_type = TEMP_VAL_CONST;
		459	+ ts->name = NULL;
		460	+ ts->val = val;
		461	+ ts++;
		462	+ ts->base_type = TCG_TYPE_I32;
		463	+ ts->type = TCG_TYPE_I32;
		464	+ ts->val_type = TEMP_VAL_CONST;
		465	+ ts->name = NULL;
		466	+ ts->val = val >> 32;
		467	+ s->nb_temps += 2;
		468	+#else
		469	+ tcg_temp_alloc(s, idx + 1);
		470	+ ts = &s->temps[idx];
		471	+ ts->base_type = ts->type = TCG_TYPE_I64;
		472	+ ts->val_type = TEMP_VAL_CONST;
		473	+ ts->name = NULL;
		474	+ ts->val = val;
		475	+ s->nb_temps++;
		476	+#endif
		477	+ return idx;
		478	+}
		479	+
		480	+void tcg_register_helper(void func, const char name)
		481	+{
		482	+ TCGContext *s = &tcg_ctx;
		483	+ int n;
		484	+ if ((s->nb_helpers + 1) > s->allocated_helpers) {
		485	+ n = s->allocated_helpers;
		486	+ if (n == 0) {
		487	+ n = 4;
		488	+ } else {
		489	+ n *= 2;
		490	+ }
		491	+ s->helpers = realloc(s->helpers, n * sizeof(TCGHelperInfo));
		492	+ s->allocated_helpers = n;
		493	+ }
		494	+ s->helpers[s->nb_helpers].func = func;
		495	+ s->helpers[s->nb_helpers].name = name;
		496	+ s->nb_helpers++;
		497	+}
		498	+
		499	+const char tcg_helper_get_name(TCGContext s, void *func)
		500	+{
		501	+ int i;
		502	+
		503	+ for(i = 0; i < s->nb_helpers; i++) {
		504	+ if (s->helpers[i].func == func)
		505	+ return s->helpers[i].name;
		506	+ }
		507	+ return NULL;
		508	+}
		509	+
		510	+static inline TCGType tcg_get_base_type(TCGContext *s, TCGArg arg)
		511	+{
		512	+ return s->temps[arg].base_type;
		513	+}
		514	+
		515	+static void tcg_gen_call_internal(TCGContext *s, TCGArg func,
		516	+ unsigned int flags,
		517	+ unsigned int nb_rets, const TCGArg *rets,
		518	+ unsigned int nb_params, const TCGArg *params)
		519	+{
		520	+ int i;
		521	+ *gen_opc_ptr++ = INDEX_op_call;
		522	+ *gen_opparam_ptr++ = (nb_rets << 16) \| (nb_params + 1);
		523	+ for(i = 0; i < nb_rets; i++) {
		524	+ *gen_opparam_ptr++ = rets[i];
		525	+ }
		526	+ for(i = 0; i < nb_params; i++) {
		527	+ *gen_opparam_ptr++ = params[i];
		528	+ }
		529	+ *gen_opparam_ptr++ = func;
		530	+
		531	+ *gen_opparam_ptr++ = flags;
		532	+ /* total parameters, needed to go backward in the instruction stream */
		533	+ *gen_opparam_ptr++ = 1 + nb_rets + nb_params + 3;
		534	+}
		535	+
		536	+
		537	+#if TCG_TARGET_REG_BITS < 64
		538	+/* Note: we convert the 64 bit args to 32 bit */
		539	+void tcg_gen_call(TCGContext *s, TCGArg func, unsigned int flags,
		540	+ unsigned int nb_rets, const TCGArg *rets,
		541	+ unsigned int nb_params, const TCGArg *args1)
		542	+{
		543	+ TCGArg ret, *args2, rets_2[2], arg;
		544	+ int j, i, call_type;
		545	+
		546	+ if (nb_rets == 1) {
		547	+ ret = rets[0];
		548	+ if (tcg_get_base_type(s, ret) == TCG_TYPE_I64) {
		549	+ nb_rets = 2;
		550	+ rets_2[0] = ret;
		551	+ rets_2[1] = ret + 1;
		552	+ rets = rets_2;
		553	+ }
		554	+ }
		555	+ args2 = alloca((nb_params * 2) * sizeof(TCGArg));
		556	+ j = 0;
		557	+ call_type = (flags & TCG_CALL_TYPE_MASK);
		558	+ for(i = 0; i < nb_params; i++) {
		559	+ arg = args1[i];
		560	+ if (tcg_get_base_type(s, arg) == TCG_TYPE_I64) {
		561	+#ifdef TCG_TARGET_I386
		562	+ /* REGPARM case: if the third parameter is 64 bit, it is
		563	+ allocated on the stack */
		564	+ if (j == 2 && call_type == TCG_CALL_TYPE_REGPARM) {
		565	+ call_type = TCG_CALL_TYPE_REGPARM_2;
		566	+ flags = (flags & ~TCG_CALL_TYPE_MASK) \| call_type;
		567	+ }
		568	+ args2[j++] = arg;
		569	+ args2[j++] = arg + 1;
		570	+#else
		571	+#ifdef TCG_TARGET_WORDS_BIGENDIAN
		572	+ args2[j++] = arg + 1;
		573	+ args2[j++] = arg;
		574	+#else
		575	+ args2[j++] = arg;
		576	+ args2[j++] = arg + 1;
		577	+#endif
		578	+#endif
		579	+ } else {
		580	+ args2[j++] = arg;
		581	+ }
		582	+ }
		583	+ tcg_gen_call_internal(s, func, flags,
		584	+ nb_rets, rets, j, args2);
		585	+}
		586	+#else
		587	+void tcg_gen_call(TCGContext *s, TCGArg func, unsigned int flags,
		588	+ unsigned int nb_rets, const TCGArg *rets,
		589	+ unsigned int nb_params, const TCGArg *args1)
		590	+{
		591	+ tcg_gen_call_internal(s, func, flags,
		592	+ nb_rets, rets, nb_params, args1);
		593	+}
		594	+#endif
		595	+
		596	+void tcg_gen_shifti_i64(TCGArg ret, TCGArg arg1,
		597	+ int c, int right, int arith)
		598	+{
		599	+ if (c == 0)
		600	+ return;
		601	+ if (c >= 32) {
		602	+ c -= 32;
		603	+ if (right) {
		604	+ if (arith) {
		605	+ tcg_gen_sari_i32(ret, arg1 + 1, c);
		606	+ tcg_gen_sari_i32(ret + 1, arg1 + 1, 31);
		607	+ } else {
		608	+ tcg_gen_shri_i32(ret, arg1 + 1, c);
		609	+ tcg_gen_movi_i32(ret + 1, 0);
		610	+ }
		611	+ } else {
		612	+ tcg_gen_shli_i32(ret + 1, arg1, c);
		613	+ tcg_gen_movi_i32(ret, 0);
		614	+ }
		615	+ } else {
		616	+ int t0, t1;
		617	+
		618	+ t0 = tcg_temp_new(TCG_TYPE_I32);
		619	+ t1 = tcg_temp_new(TCG_TYPE_I32);
		620	+ if (right) {
		621	+ tcg_gen_shli_i32(t0, arg1 + 1, 32 - c);
		622	+ if (arith)
		623	+ tcg_gen_sari_i32(t1, arg1 + 1, c);
		624	+ else
		625	+ tcg_gen_shri_i32(t1, arg1 + 1, c);
		626	+ tcg_gen_shri_i32(ret, arg1, c);
		627	+ tcg_gen_or_i32(ret, ret, t0);
		628	+ tcg_gen_mov_i32(ret + 1, t1);
		629	+ } else {
		630	+ tcg_gen_shri_i32(t0, arg1, 32 - c);
		631	+ /* Note: ret can be the same as arg1, so we use t1 */
		632	+ tcg_gen_shli_i32(t1, arg1, c);
		633	+ tcg_gen_shli_i32(ret + 1, arg1 + 1, c);
		634	+ tcg_gen_or_i32(ret + 1, ret + 1, t0);
		635	+ tcg_gen_mov_i32(ret, t1);
		636	+ }
		637	+ }
		638	+}
		639	+
		640	+void tcg_reg_alloc_start(TCGContext *s)
		641	+{
		642	+ int i;
		643	+ TCGTemp *ts;
		644	+ for(i = 0; i < s->nb_globals; i++) {
		645	+ ts = &s->temps[i];
		646	+ if (ts->fixed_reg) {
		647	+ ts->val_type = TEMP_VAL_REG;
		648	+ } else {
		649	+ ts->val_type = TEMP_VAL_MEM;
		650	+ }
		651	+ }
		652	+ for(i = 0; i < TCG_TARGET_NB_REGS; i++) {
		653	+ s->reg_to_temp[i] = -1;
		654	+ }
		655	+}
		656	+
		657	+char tcg_get_arg_str(TCGContext s, char *buf, int buf_size, TCGArg arg)
		658	+{
		659	+ TCGTemp *ts;
		660	+ if (arg < s->nb_globals) {
		661	+ pstrcpy(buf, buf_size, s->temps[arg].name);
		662	+ } else {
		663	+ ts = &s->temps[arg];
		664	+ if (ts->val_type == TEMP_VAL_CONST) {
		665	+ snprintf(buf, buf_size, "$0x%" TCG_PRIlx , ts->val);
		666	+ } else {
		667	+ snprintf(buf, buf_size, "tmp%d", (int)arg - s->nb_globals);
		668	+ }
		669	+ }
		670	+ return buf;
		671	+}
		672	+
		673	+void tcg_dump_ops(TCGContext s, FILE outfile)
		674	+{
		675	+ const uint16_t *opc_ptr;
		676	+ const TCGArg *args;
		677	+ TCGArg arg;
		678	+ int c, i, k, nb_oargs, nb_iargs, nb_cargs;
		679	+ const TCGOpDef *def;
		680	+ char buf[128];
		681	+
		682	+ opc_ptr = gen_opc_buf;
		683	+ args = gen_opparam_buf;
		684	+ while (opc_ptr < gen_opc_ptr) {
		685	+ c = *opc_ptr++;
		686	+ def = &tcg_op_defs[c];
		687	+ fprintf(outfile, " %s ", def->name);
		688	+ if (c == INDEX_op_call) {
		689	+ TCGArg arg;
		690	+ /* variable number of arguments */
		691	+ arg = *args++;
		692	+ nb_oargs = arg >> 16;
		693	+ nb_iargs = arg & 0xffff;
		694	+ nb_cargs = def->nb_cargs;
		695	+ } else if (c == INDEX_op_nopn) {
		696	+ /* variable number of arguments */
		697	+ nb_cargs = *args;
		698	+ nb_oargs = 0;
		699	+ nb_iargs = 0;
		700	+ } else {
		701	+ nb_oargs = def->nb_oargs;
		702	+ nb_iargs = def->nb_iargs;
		703	+ nb_cargs = def->nb_cargs;
		704	+ }
		705	+
		706	+ k = 0;
		707	+ for(i = 0; i < nb_oargs; i++) {
		708	+ if (k != 0)
		709	+ fprintf(outfile, ",");
		710	+ fprintf(outfile, "%s", tcg_get_arg_str(s, buf, sizeof(buf), args[k++]));
		711	+ }
		712	+ for(i = 0; i < nb_iargs; i++) {
		713	+ if (k != 0)
		714	+ fprintf(outfile, ",");
		715	+ /* XXX: dump helper name for call */
		716	+ fprintf(outfile, "%s", tcg_get_arg_str(s, buf, sizeof(buf), args[k++]));
		717	+ }
		718	+ for(i = 0; i < nb_cargs; i++) {
		719	+ if (k != 0)
		720	+ fprintf(outfile, ",");
		721	+ arg = args[k++];
		722	+ fprintf(outfile, "$0x%" TCG_PRIlx, arg);
		723	+ }
		724	+ fprintf(outfile, "\n");
		725	+ args += nb_iargs + nb_oargs + nb_cargs;
		726	+ }
		727	+}
		728	+
		729	+/* we give more priority to constraints with less registers */
		730	+static int get_constraint_priority(const TCGOpDef *def, int k)
		731	+{
		732	+ const TCGArgConstraint *arg_ct;
		733	+
		734	+ int i, n;
		735	+ arg_ct = &def->args_ct[k];
		736	+ if (arg_ct->ct & TCG_CT_ALIAS) {
		737	+ /* an alias is equivalent to a single register */
		738	+ n = 1;
		739	+ } else {
		740	+ if (!(arg_ct->ct & TCG_CT_REG))
		741	+ return 0;
		742	+ n = 0;
		743	+ for(i = 0; i < TCG_TARGET_NB_REGS; i++) {
		744	+ if (tcg_regset_test_reg(arg_ct->u.regs, i))
		745	+ n++;
		746	+ }
		747	+ }
		748	+ return TCG_TARGET_NB_REGS - n + 1;
		749	+}
		750	+
		751	+/* sort from highest priority to lowest */
		752	+static void sort_constraints(TCGOpDef *def, int start, int n)
		753	+{
		754	+ int i, j, p1, p2, tmp;
		755	+
		756	+ for(i = 0; i < n; i++)
		757	+ def->sorted_args[start + i] = start + i;
		758	+ if (n <= 1)
		759	+ return;
		760	+ for(i = 0; i < n - 1; i++) {
		761	+ for(j = i + 1; j < n; j++) {
		762	+ p1 = get_constraint_priority(def, def->sorted_args[start + i]);
		763	+ p2 = get_constraint_priority(def, def->sorted_args[start + j]);
		764	+ if (p1 < p2) {
		765	+ tmp = def->sorted_args[start + i];
		766	+ def->sorted_args[start + i] = def->sorted_args[start + j];
		767	+ def->sorted_args[start + j] = tmp;
		768	+ }
		769	+ }
		770	+ }
		771	+}
		772	+
		773	+void tcg_add_target_add_op_defs(const TCGTargetOpDef *tdefs)
		774	+{
		775	+ int op;
		776	+ TCGOpDef *def;
		777	+ const char *ct_str;
		778	+ int i, nb_args;
		779	+
		780	+ for(;;) {
		781	+ if (tdefs->op < 0)
		782	+ break;
		783	+ op = tdefs->op;
		784	+ assert(op >= 0 && op < NB_OPS);
		785	+ def = &tcg_op_defs[op];
		786	+ nb_args = def->nb_iargs + def->nb_oargs;
		787	+ for(i = 0; i < nb_args; i++) {
		788	+ ct_str = tdefs->args_ct_str[i];
		789	+ tcg_regset_clear(def->args_ct[i].u.regs);
		790	+ def->args_ct[i].ct = 0;
		791	+ if (ct_str[0] >= '0' && ct_str[0] <= '9') {
		792	+ int oarg;
		793	+ oarg = ct_str[0] - '0';
		794	+ assert(oarg < def->nb_oargs);
		795	+ assert(def->args_ct[oarg].ct & TCG_CT_REG);
		796	+ /* TCG_CT_ALIAS is for the output arguments. The input
		797	+ argument is tagged with TCG_CT_IALIAS for
		798	+ informative purposes. */
		799	+ def->args_ct[i] = def->args_ct[oarg];
		800	+ def->args_ct[oarg].ct = i \| TCG_CT_ALIAS;
		801	+ def->args_ct[i].ct \|= TCG_CT_IALIAS;
		802	+ } else {
		803	+ for(;;) {
		804	+ if (*ct_str == '\0')
		805	+ break;
		806	+ switch(*ct_str) {
		807	+ case 'i':
		808	+ def->args_ct[i].ct \|= TCG_CT_CONST;
		809	+ ct_str++;
		810	+ break;
		811	+ default:
		812	+ if (target_parse_constraint(&def->args_ct[i], &ct_str) < 0) {
		813	+ fprintf(stderr, "Invalid constraint '%s' for arg %d of operation '%s'\n",
		814	+ ct_str, i, def->name);
		815	+ exit(1);
		816	+ }
		817	+ }
		818	+ }
		819	+ }
		820	+ }
		821	+
		822	+ /* sort the constraints (XXX: this is just an heuristic) */
		823	+ sort_constraints(def, 0, def->nb_oargs);
		824	+ sort_constraints(def, def->nb_oargs, def->nb_iargs);
		825	+
		826	+#if 0
		827	+ {
		828	+ int i;
		829	+
		830	+ printf("%s: sorted=", def->name);
		831	+ for(i = 0; i < def->nb_oargs + def->nb_iargs; i++)
		832	+ printf(" %d", def->sorted_args[i]);
		833	+ printf("\n");
		834	+ }
		835	+#endif
		836	+ tdefs++;
		837	+ }
		838	+
		839	+}
		840	+
		841	+#ifdef USE_LIVENESS_ANALYSIS
		842	+
		843	+/* set a nop for an operation using 'nb_args' */
		844	+static inline void tcg_set_nop(TCGContext s, uint16_t opc_ptr,
		845	+ TCGArg *args, int nb_args)
		846	+{
		847	+ if (nb_args == 0) {
		848	+ *opc_ptr = INDEX_op_nop;
		849	+ } else {
		850	+ *opc_ptr = INDEX_op_nopn;
		851	+ args[0] = nb_args;
		852	+ args[nb_args - 1] = nb_args;
		853	+ }
		854	+}
		855	+
		856	+/* liveness analysis: end of basic block: globals are live, temps are dead */
		857	+static inline void tcg_la_bb_end(TCGContext s, uint8_t dead_temps)
		858	+{
		859	+ memset(dead_temps, 0, s->nb_globals);
		860	+ memset(dead_temps + s->nb_globals, 1, s->nb_temps - s->nb_globals);
		861	+}
		862	+
		863	+/* Liveness analysis : update the opc_dead_iargs array to tell if a
		864	+ given input arguments is dead. Instructions updating dead
		865	+ temporaries are removed. */
		866	+void tcg_liveness_analysis(TCGContext *s)
		867	+{
		868	+ int i, op_index, op, nb_args, nb_iargs, nb_oargs, arg, nb_ops;
		869	+ TCGArg *args;
		870	+ const TCGOpDef *def;
		871	+ uint8_t *dead_temps;
		872	+ unsigned int dead_iargs;
		873	+
		874	+ gen_opc_ptr++; /* skip end */
		875	+
		876	+ nb_ops = gen_opc_ptr - gen_opc_buf;
		877	+
		878	+ /* XXX: make it really dynamic */
		879	+ s->op_dead_iargs = tcg_malloc(OPC_BUF_SIZE * sizeof(uint16_t));
		880	+
		881	+ dead_temps = tcg_malloc(s->nb_temps);
		882	+ memset(dead_temps, 1, s->nb_temps);
		883	+
		884	+ args = gen_opparam_ptr;
		885	+ op_index = nb_ops - 1;
		886	+ while (op_index >= 0) {
		887	+ op = gen_opc_buf[op_index];
		888	+ def = &tcg_op_defs[op];
		889	+ switch(op) {
		890	+ case INDEX_op_call:
		891	+ nb_args = args[-1];
		892	+ args -= nb_args;
		893	+ nb_iargs = args[0] & 0xffff;
		894	+ nb_oargs = args[0] >> 16;
		895	+ args++;
		896	+
		897	+ /* output args are dead */
		898	+ for(i = 0; i < nb_oargs; i++) {
		899	+ arg = args[i];
		900	+ dead_temps[arg] = 1;
		901	+ }
		902	+
		903	+ /* globals are live (they may be used by the call) */
		904	+ memset(dead_temps, 0, s->nb_globals);
		905	+
		906	+ /* input args are live */
		907	+ dead_iargs = 0;
		908	+ for(i = 0; i < nb_iargs; i++) {
		909	+ arg = args[i + nb_oargs];
		910	+ if (dead_temps[arg]) {
		911	+ dead_iargs \|= (1 << i);
		912	+ }
		913	+ dead_temps[arg] = 0;
		914	+ }
		915	+ s->op_dead_iargs[op_index] = dead_iargs;
		916	+ args--;
		917	+ break;
		918	+ case INDEX_op_set_label:
		919	+ args--;
		920	+ /* mark end of basic block */
		921	+ tcg_la_bb_end(s, dead_temps);
		922	+ break;
		923	+ case INDEX_op_nopn:
		924	+ nb_args = args[-1];
		925	+ args -= nb_args;
		926	+ break;
		927	+ case INDEX_op_macro_2:
		928	+ {
		929	+ int dead_args[2], macro_id;
		930	+ int saved_op_index, saved_arg_index;
		931	+ int macro_op_index, macro_arg_index;
		932	+ int macro_end_op_index, macro_end_arg_index;
		933	+ int last_nb_temps;
		934	+
		935	+ nb_args = 3;
		936	+ args -= nb_args;
		937	+ dead_args[0] = dead_temps[args[0]];
		938	+ dead_args[1] = dead_temps[args[1]];
		939	+ macro_id = args[2];
		940	+
		941	+ /* call the macro function which generate code
		942	+ depending on the live outputs */
		943	+ saved_op_index = op_index;
		944	+ saved_arg_index = args - gen_opparam_buf;
		945	+
		946	+ /* add a macro start instruction */
		947	+ *gen_opc_ptr++ = INDEX_op_macro_start;
		948	+ *gen_opparam_ptr++ = saved_op_index;
		949	+ *gen_opparam_ptr++ = saved_arg_index;
		950	+
		951	+ macro_op_index = gen_opc_ptr - gen_opc_buf;
		952	+ macro_arg_index = gen_opparam_ptr - gen_opparam_buf;
		953	+
		954	+ last_nb_temps = s->nb_temps;
		955	+
		956	+ s->macro_func(s, macro_id, dead_args);
		957	+
		958	+ /* realloc temp info (XXX: make it faster) */
		959	+ if (s->nb_temps > last_nb_temps) {
		960	+ uint8_t *new_dead_temps;
		961	+
		962	+ new_dead_temps = tcg_malloc(s->nb_temps);
		963	+ memcpy(new_dead_temps, dead_temps, last_nb_temps);
		964	+ memset(new_dead_temps + last_nb_temps, 1,
		965	+ s->nb_temps - last_nb_temps);
		966	+ dead_temps = new_dead_temps;
		967	+ }
		968	+
		969	+ macro_end_op_index = gen_opc_ptr - gen_opc_buf;
		970	+ macro_end_arg_index = gen_opparam_ptr - gen_opparam_buf;
		971	+
		972	+ /* end of macro: add a goto to the next instruction */
		973	+ *gen_opc_ptr++ = INDEX_op_macro_end;
		974	+ *gen_opparam_ptr++ = op_index + 1;
		975	+ *gen_opparam_ptr++ = saved_arg_index + nb_args;
		976	+
		977	+ /* modify the macro operation to be a macro_goto */
		978	+ gen_opc_buf[op_index] = INDEX_op_macro_goto;
		979	+ args[0] = macro_op_index;
		980	+ args[1] = macro_arg_index;
		981	+ args[2] = 0; /* dummy third arg to match the
		982	+ macro parameters */
		983	+
		984	+ /* set the next instruction to the end of the macro */
		985	+ op_index = macro_end_op_index;
		986	+ args = macro_end_arg_index + gen_opparam_buf;
		987	+ }
		988	+ break;
		989	+ case INDEX_op_macro_start:
		990	+ args -= 2;
		991	+ op_index = args[0];
		992	+ args = gen_opparam_buf + args[1];
		993	+ break;
		994	+ case INDEX_op_macro_goto:
		995	+ case INDEX_op_macro_end:
		996	+ tcg_abort(); /* should never happen in liveness analysis */
		997	+ case INDEX_op_end:
		998	+ break;
		999	+ /* XXX: optimize by hardcoding common cases (e.g. triadic ops) */
		1000	+ default:
		1001	+ if (op > INDEX_op_end) {
		1002	+ args -= def->nb_args;
		1003	+ nb_iargs = def->nb_iargs;
		1004	+ nb_oargs = def->nb_oargs;
		1005	+
		1006	+ /* Test if the operation can be removed because all
		1007	+ its outputs are dead. We may add a flag to
		1008	+ explicitely tell if the op has side
		1009	+ effects. Currently we assume that if nb_oargs == 0
		1010	+ or OPF_BB_END is set, the operation has side
		1011	+ effects and cannot be removed */
		1012	+ if (nb_oargs != 0 && !(def->flags & TCG_OPF_BB_END)) {
		1013	+ for(i = 0; i < nb_oargs; i++) {
		1014	+ arg = args[i];
		1015	+ if (!dead_temps[arg])
		1016	+ goto do_not_remove;
		1017	+ }
		1018	+ tcg_set_nop(s, gen_opc_buf + op_index, args, def->nb_args);
		1019	+#ifdef CONFIG_PROFILER
		1020	+ {
		1021	+ extern int64_t dyngen_tcg_del_op_count;
		1022	+ dyngen_tcg_del_op_count++;
		1023	+ }
		1024	+#endif
		1025	+ } else {
		1026	+ do_not_remove:
		1027	+
		1028	+ /* output args are dead */
		1029	+ for(i = 0; i < nb_oargs; i++) {
		1030	+ arg = args[i];
		1031	+ dead_temps[arg] = 1;
		1032	+ }
		1033	+
		1034	+ /* if end of basic block, update */
		1035	+ if (def->flags & TCG_OPF_BB_END) {
		1036	+ tcg_la_bb_end(s, dead_temps);
		1037	+ }
		1038	+
		1039	+ /* input args are live */
		1040	+ dead_iargs = 0;
		1041	+ for(i = 0; i < nb_iargs; i++) {
		1042	+ arg = args[i + nb_oargs];
		1043	+ if (dead_temps[arg]) {
		1044	+ dead_iargs \|= (1 << i);
		1045	+ }
		1046	+ dead_temps[arg] = 0;
		1047	+ }
		1048	+ s->op_dead_iargs[op_index] = dead_iargs;
		1049	+ }
		1050	+ } else {
		1051	+ /* legacy dyngen operations */
		1052	+ args -= def->nb_args;
		1053	+ /* mark end of basic block */
		1054	+ tcg_la_bb_end(s, dead_temps);
		1055	+ }
		1056	+ break;
		1057	+ }
		1058	+ op_index--;
		1059	+ }
		1060	+
		1061	+ if (args != gen_opparam_buf)
		1062	+ tcg_abort();
		1063	+}
		1064	+#else
		1065	+/* dummy liveness analysis */
		1066	+void tcg_liveness_analysis(TCGContext *s)
		1067	+{
		1068	+ int nb_ops;
		1069	+ nb_ops = gen_opc_ptr - gen_opc_buf;
		1070	+
		1071	+ s->op_dead_iargs = tcg_malloc(nb_ops * sizeof(uint16_t));
		1072	+ memset(s->op_dead_iargs, 0, nb_ops * sizeof(uint16_t));
		1073	+}
		1074	+#endif
		1075	+
		1076	+#ifndef NDEBUG
		1077	+static void dump_regs(TCGContext *s)
		1078	+{
		1079	+ TCGTemp *ts;
		1080	+ int i;
		1081	+ char buf[64];
		1082	+
		1083	+ for(i = 0; i < s->nb_temps; i++) {
		1084	+ ts = &s->temps[i];
		1085	+ printf(" %10s: ", tcg_get_arg_str(s, buf, sizeof(buf), i));
		1086	+ switch(ts->val_type) {
		1087	+ case TEMP_VAL_REG:
		1088	+ printf("%s", tcg_target_reg_names[ts->reg]);
		1089	+ break;
		1090	+ case TEMP_VAL_MEM:
		1091	+ printf("%d(%s)", (int)ts->mem_offset, tcg_target_reg_names[ts->mem_reg]);
		1092	+ break;
		1093	+ case TEMP_VAL_CONST:
		1094	+ printf("$0x%" TCG_PRIlx, ts->val);
		1095	+ break;
		1096	+ case TEMP_VAL_DEAD:
		1097	+ printf("D");
		1098	+ break;
		1099	+ default:
		1100	+ printf("???");
		1101	+ break;
		1102	+ }
		1103	+ printf("\n");
		1104	+ }
		1105	+
		1106	+ for(i = 0; i < TCG_TARGET_NB_REGS; i++) {
		1107	+ if (s->reg_to_temp[i] >= 0) {
		1108	+ printf("%s: %s\n",
		1109	+ tcg_target_reg_names[i],
		1110	+ tcg_get_arg_str(s, buf, sizeof(buf), s->reg_to_temp[i]));
		1111	+ }
		1112	+ }
		1113	+}
		1114	+
		1115	+static void check_regs(TCGContext *s)
		1116	+{
		1117	+ int reg, k;
		1118	+ TCGTemp *ts;
		1119	+ char buf[64];
		1120	+
		1121	+ for(reg = 0; reg < TCG_TARGET_NB_REGS; reg++) {
		1122	+ k = s->reg_to_temp[reg];
		1123	+ if (k >= 0) {
		1124	+ ts = &s->temps[k];
		1125	+ if (ts->val_type != TEMP_VAL_REG \|\|
		1126	+ ts->reg != reg) {
		1127	+ printf("Inconsistency for register %s:\n",
		1128	+ tcg_target_reg_names[reg]);
		1129	+ printf("reg state:\n");
		1130	+ dump_regs(s);
		1131	+ tcg_abort();
		1132	+ }
		1133	+ }
		1134	+ }
		1135	+ for(k = 0; k < s->nb_temps; k++) {
		1136	+ ts = &s->temps[k];
		1137	+ if (ts->val_type == TEMP_VAL_REG &&
		1138	+ !ts->fixed_reg &&
		1139	+ s->reg_to_temp[ts->reg] != k) {
		1140	+ printf("Inconsistency for temp %s:\n",
		1141	+ tcg_get_arg_str(s, buf, sizeof(buf), k));
		1142	+ printf("reg state:\n");
		1143	+ dump_regs(s);
		1144	+ tcg_abort();
		1145	+ }
		1146	+ }
		1147	+}
		1148	+#endif
		1149	+
		1150	+static void temp_allocate_frame(TCGContext *s, int temp)
		1151	+{
		1152	+ TCGTemp *ts;
		1153	+ ts = &s->temps[temp];
		1154	+ s->current_frame_offset = (s->current_frame_offset + sizeof(tcg_target_long) - 1) & ~(sizeof(tcg_target_long) - 1);
		1155	+ if (s->current_frame_offset + sizeof(tcg_target_long) > s->frame_end)
		1156	+ abort();
		1157	+ ts->mem_offset = s->current_frame_offset;
		1158	+ ts->mem_reg = s->frame_reg;
		1159	+ ts->mem_allocated = 1;
		1160	+ s->current_frame_offset += sizeof(tcg_target_long);
		1161	+}
		1162	+
		1163	+/* free register 'reg' by spilling the corresponding temporary if necessary */
		1164	+static void tcg_reg_free(TCGContext *s, int reg)
		1165	+{
		1166	+ TCGTemp *ts;
		1167	+ int temp;
		1168	+
		1169	+ temp = s->reg_to_temp[reg];
		1170	+ if (temp != -1) {
		1171	+ ts = &s->temps[temp];
		1172	+ assert(ts->val_type == TEMP_VAL_REG);
		1173	+ if (!ts->mem_coherent) {
		1174	+ if (!ts->mem_allocated)
		1175	+ temp_allocate_frame(s, temp);
		1176	+ tcg_out_st(s, reg, ts->mem_reg, ts->mem_offset);
		1177	+ }
		1178	+ ts->val_type = TEMP_VAL_MEM;
		1179	+ s->reg_to_temp[reg] = -1;
		1180	+ }
		1181	+}
		1182	+
		1183	+/* Allocate a register belonging to reg1 & ~reg2 */
		1184	+static int tcg_reg_alloc(TCGContext *s, TCGRegSet reg1, TCGRegSet reg2)
		1185	+{
		1186	+ int i, reg;
		1187	+ TCGRegSet reg_ct;
		1188	+
		1189	+ tcg_regset_andnot(reg_ct, reg1, reg2);
		1190	+
		1191	+ /* first try free registers */
		1192	+ for(i = 0; i < TCG_TARGET_NB_REGS; i++) {
		1193	+ reg = tcg_target_reg_alloc_order[i];
		1194	+ if (tcg_regset_test_reg(reg_ct, reg) && s->reg_to_temp[reg] == -1)
		1195	+ return reg;
		1196	+ }
		1197	+
		1198	+ /* XXX: do better spill choice */
		1199	+ for(i = 0; i < TCG_TARGET_NB_REGS; i++) {
		1200	+ reg = tcg_target_reg_alloc_order[i];
		1201	+ if (tcg_regset_test_reg(reg_ct, reg)) {
		1202	+ tcg_reg_free(s, reg);
		1203	+ return reg;
		1204	+ }
		1205	+ }
		1206	+
		1207	+ tcg_abort();
		1208	+}
		1209	+
		1210	+/* at the end of a basic block, we assume all temporaries are dead and
		1211	+ all globals are stored at their canonical location */
		1212	+/* XXX: optimize by handling constants in another array ? */
		1213	+void tcg_reg_alloc_bb_end(TCGContext *s)
		1214	+{
		1215	+ TCGTemp *ts;
		1216	+ int i;
		1217	+
		1218	+ for(i = 0; i < s->nb_globals; i++) {
		1219	+ ts = &s->temps[i];
		1220	+ if (!ts->fixed_reg) {
		1221	+ if (ts->val_type == TEMP_VAL_REG) {
		1222	+ tcg_reg_free(s, ts->reg);
		1223	+ }
		1224	+ }
		1225	+ }
		1226	+
		1227	+ for(i = s->nb_globals; i < s->nb_temps; i++) {
		1228	+ ts = &s->temps[i];
		1229	+ if (ts->val_type != TEMP_VAL_CONST) {
		1230	+ if (ts->val_type == TEMP_VAL_REG) {
		1231	+ s->reg_to_temp[ts->reg] = -1;
		1232	+ }
		1233	+ ts->val_type = TEMP_VAL_DEAD;
		1234	+ }
		1235	+ }
		1236	+}
		1237	+
		1238	+#define IS_DEAD_IARG(n) ((dead_iargs >> (n)) & 1)
		1239	+
		1240	+static void tcg_reg_alloc_mov(TCGContext s, const TCGOpDef def,
		1241	+ const TCGArg *args,
		1242	+ unsigned int dead_iargs)
		1243	+{
		1244	+ TCGTemp ts, ots;
		1245	+ int reg;
		1246	+ const TCGArgConstraint *arg_ct;
		1247	+
		1248	+ ots = &s->temps[args[0]];
		1249	+ ts = &s->temps[args[1]];
		1250	+ arg_ct = &def->args_ct[0];
		1251	+
		1252	+ if (ts->val_type == TEMP_VAL_REG) {
		1253	+ if (IS_DEAD_IARG(0) && !ts->fixed_reg && !ots->fixed_reg) {
		1254	+ /* the mov can be suppressed */
		1255	+ if (ots->val_type == TEMP_VAL_REG)
		1256	+ s->reg_to_temp[ots->reg] = -1;
		1257	+ reg = ts->reg;
		1258	+ s->reg_to_temp[reg] = -1;
		1259	+ ts->val_type = TEMP_VAL_DEAD;
		1260	+ } else {
		1261	+ if (ots->val_type == TEMP_VAL_REG) {
		1262	+ reg = ots->reg;
		1263	+ } else {
		1264	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, s->reserved_regs);
		1265	+ }
		1266	+ if (ts->reg != reg) {
		1267	+ tcg_out_mov(s, reg, ts->reg);
		1268	+ }
		1269	+ }
		1270	+ } else if (ts->val_type == TEMP_VAL_MEM) {
		1271	+ if (ots->val_type == TEMP_VAL_REG) {
		1272	+ reg = ots->reg;
		1273	+ } else {
		1274	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, s->reserved_regs);
		1275	+ }
		1276	+ tcg_out_ld(s, reg, ts->mem_reg, ts->mem_offset);
		1277	+ } else if (ts->val_type == TEMP_VAL_CONST) {
		1278	+ if (ots->val_type == TEMP_VAL_REG) {
		1279	+ reg = ots->reg;
		1280	+ } else {
		1281	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, s->reserved_regs);
		1282	+ }
		1283	+ tcg_out_movi(s, ots->type, reg, ts->val);
		1284	+ } else {
		1285	+ tcg_abort();
		1286	+ }
		1287	+ s->reg_to_temp[reg] = args[0];
		1288	+ ots->reg = reg;
		1289	+ ots->val_type = TEMP_VAL_REG;
		1290	+ ots->mem_coherent = 0;
		1291	+}
		1292	+
		1293	+static void tcg_reg_alloc_op(TCGContext *s,
		1294	+ const TCGOpDef *def, int opc,
		1295	+ const TCGArg *args,
		1296	+ unsigned int dead_iargs)
		1297	+{
		1298	+ TCGRegSet allocated_regs;
		1299	+ int i, k, nb_iargs, nb_oargs, reg;
		1300	+ TCGArg arg;
		1301	+ const TCGArgConstraint *arg_ct;
		1302	+ TCGTemp *ts;
		1303	+ TCGArg new_args[TCG_MAX_OP_ARGS];
		1304	+ int const_args[TCG_MAX_OP_ARGS];
		1305	+
		1306	+ nb_oargs = def->nb_oargs;
		1307	+ nb_iargs = def->nb_iargs;
		1308	+
		1309	+ /* copy constants */
		1310	+ memcpy(new_args + nb_oargs + nb_iargs,
		1311	+ args + nb_oargs + nb_iargs,
		1312	+ sizeof(TCGArg) * def->nb_cargs);
		1313	+
		1314	+ /* satisfy input constraints */
		1315	+ tcg_regset_set(allocated_regs, s->reserved_regs);
		1316	+ for(k = 0; k < nb_iargs; k++) {
		1317	+ i = def->sorted_args[nb_oargs + k];
		1318	+ arg = args[i];
		1319	+ arg_ct = &def->args_ct[i];
		1320	+ ts = &s->temps[arg];
		1321	+ if (ts->val_type == TEMP_VAL_MEM) {
		1322	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1323	+ tcg_out_ld(s, reg, ts->mem_reg, ts->mem_offset);
		1324	+ ts->val_type = TEMP_VAL_REG;
		1325	+ ts->reg = reg;
		1326	+ ts->mem_coherent = 1;
		1327	+ s->reg_to_temp[reg] = arg;
		1328	+ } else if (ts->val_type == TEMP_VAL_CONST) {
		1329	+ if (tcg_target_const_match(ts->val, arg_ct)) {
		1330	+ /* constant is OK for instruction */
		1331	+ const_args[i] = 1;
		1332	+ new_args[i] = ts->val;
		1333	+ goto iarg_end;
		1334	+ } else {
		1335	+ /* need to move to a register*/
		1336	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1337	+ tcg_out_movi(s, ts->type, reg, ts->val);
		1338	+ goto iarg_end1;
		1339	+ }
		1340	+ }
		1341	+ assert(ts->val_type == TEMP_VAL_REG);
		1342	+ if ((arg_ct->ct & TCG_CT_IALIAS) &&
		1343	+ !IS_DEAD_IARG(i - nb_oargs)) {
		1344	+ /* if the input is aliased to an output and if it is
		1345	+ not dead after the instruction, we must allocate
		1346	+ a new register and move it */
		1347	+ goto allocate_in_reg;
		1348	+ }
		1349	+ reg = ts->reg;
		1350	+ if (tcg_regset_test_reg(arg_ct->u.regs, reg)) {
		1351	+ /* nothing to do : the constraint is satisfied */
		1352	+ } else {
		1353	+ allocate_in_reg:
		1354	+ /* allocate a new register matching the constraint
		1355	+ and move the temporary register into it */
		1356	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1357	+ tcg_out_mov(s, reg, ts->reg);
		1358	+ }
		1359	+ iarg_end1:
		1360	+ new_args[i] = reg;
		1361	+ const_args[i] = 0;
		1362	+ tcg_regset_set_reg(allocated_regs, reg);
		1363	+ iarg_end: ;
		1364	+ }
		1365	+
		1366	+ /* mark dead temporaries and free the associated registers */
		1367	+ for(i = 0; i < nb_iargs; i++) {
		1368	+ arg = args[nb_oargs + i];
		1369	+ if (IS_DEAD_IARG(i)) {
		1370	+ ts = &s->temps[arg];
		1371	+ if (ts->val_type != TEMP_VAL_CONST && !ts->fixed_reg) {
		1372	+ if (ts->val_type == TEMP_VAL_REG)
		1373	+ s->reg_to_temp[ts->reg] = -1;
		1374	+ ts->val_type = TEMP_VAL_DEAD;
		1375	+ }
		1376	+ }
		1377	+ }
		1378	+
		1379	+ /* XXX: permit generic clobber register list ? */
		1380	+ if (def->flags & TCG_OPF_CALL_CLOBBER) {
		1381	+ for(reg = 0; reg < TCG_TARGET_NB_REGS; reg++) {
		1382	+ if (tcg_regset_test_reg(tcg_target_call_clobber_regs, reg)) {
		1383	+ tcg_reg_free(s, reg);
		1384	+ }
		1385	+ }
		1386	+ }
		1387	+
		1388	+ /* satisfy the output constraints */
		1389	+ tcg_regset_set(allocated_regs, s->reserved_regs);
		1390	+ for(k = 0; k < nb_oargs; k++) {
		1391	+ i = def->sorted_args[k];
		1392	+ arg = args[i];
		1393	+ arg_ct = &def->args_ct[i];
		1394	+ ts = &s->temps[arg];
		1395	+ if (arg_ct->ct & TCG_CT_ALIAS) {
		1396	+ reg = new_args[arg_ct->ct & ~TCG_CT_ALIAS];
		1397	+ } else {
		1398	+ /* if fixed register, we try to use it */
		1399	+ reg = ts->reg;
		1400	+ if (ts->fixed_reg &&
		1401	+ tcg_regset_test_reg(arg_ct->u.regs, reg)) {
		1402	+ goto oarg_end;
		1403	+ }
		1404	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1405	+ }
		1406	+ tcg_regset_set_reg(allocated_regs, reg);
		1407	+ /* if a fixed register is used, then a move will be done afterwards */
		1408	+ if (!ts->fixed_reg) {
		1409	+ if (ts->val_type == TEMP_VAL_REG)
		1410	+ s->reg_to_temp[ts->reg] = -1;
		1411	+ ts->val_type = TEMP_VAL_REG;
		1412	+ ts->reg = reg;
		1413	+ /* temp value is modified, so the value kept in memory is
		1414	+ potentially not the same */
		1415	+ ts->mem_coherent = 0;
		1416	+ s->reg_to_temp[reg] = arg;
		1417	+ }
		1418	+ oarg_end:
		1419	+ new_args[i] = reg;
		1420	+ }
		1421	+
		1422	+ if (def->flags & TCG_OPF_BB_END)
		1423	+ tcg_reg_alloc_bb_end(s);
		1424	+
		1425	+ /* emit instruction */
		1426	+ tcg_out_op(s, opc, new_args, const_args);
		1427	+
		1428	+ /* move the outputs in the correct register if needed */
		1429	+ for(i = 0; i < nb_oargs; i++) {
		1430	+ ts = &s->temps[args[i]];
		1431	+ reg = new_args[i];
		1432	+ if (ts->fixed_reg && ts->reg != reg) {
		1433	+ tcg_out_mov(s, ts->reg, reg);
		1434	+ }
		1435	+ }
		1436	+}
		1437	+
		1438	+static int tcg_reg_alloc_call(TCGContext s, const TCGOpDef def,
		1439	+ int opc, const TCGArg *args,
		1440	+ unsigned int dead_iargs)
		1441	+{
		1442	+ int nb_iargs, nb_oargs, flags, nb_regs, i, reg, nb_params;
		1443	+ TCGArg arg, func_arg;
		1444	+ TCGTemp *ts;
		1445	+ tcg_target_long stack_offset, call_stack_size;
		1446	+ int const_func_arg;
		1447	+ TCGRegSet allocated_regs;
		1448	+ const TCGArgConstraint *arg_ct;
		1449	+
		1450	+ arg = *args++;
		1451	+
		1452	+ nb_oargs = arg >> 16;
		1453	+ nb_iargs = arg & 0xffff;
		1454	+ nb_params = nb_iargs - 1;
		1455	+
		1456	+ flags = args[nb_oargs + nb_iargs];
		1457	+
		1458	+ nb_regs = tcg_target_get_call_iarg_regs_count(flags);
		1459	+ if (nb_regs > nb_params)
		1460	+ nb_regs = nb_params;
		1461	+
		1462	+ /* assign stack slots first */
		1463	+ /* XXX: preallocate call stack */
		1464	+ call_stack_size = (nb_params - nb_regs) * sizeof(tcg_target_long);
		1465	+ call_stack_size = (call_stack_size + TCG_TARGET_STACK_ALIGN - 1) &
		1466	+ ~(TCG_TARGET_STACK_ALIGN - 1);
		1467	+ tcg_out_addi(s, TCG_REG_CALL_STACK, -call_stack_size);
		1468	+
		1469	+ stack_offset = 0;
		1470	+ for(i = nb_regs; i < nb_params; i++) {
		1471	+ arg = args[nb_oargs + i];
		1472	+ ts = &s->temps[arg];
		1473	+ if (ts->val_type == TEMP_VAL_REG) {
		1474	+ tcg_out_st(s, ts->reg, TCG_REG_CALL_STACK, stack_offset);
		1475	+ } else if (ts->val_type == TEMP_VAL_MEM) {
		1476	+ reg = tcg_reg_alloc(s, tcg_target_available_regs[ts->type],
		1477	+ s->reserved_regs);
		1478	+ /* XXX: not correct if reading values from the stack */
		1479	+ tcg_out_ld(s, reg, ts->mem_reg, ts->mem_offset);
		1480	+ tcg_out_st(s, reg, TCG_REG_CALL_STACK, stack_offset);
		1481	+ } else if (ts->val_type == TEMP_VAL_CONST) {
		1482	+ reg = tcg_reg_alloc(s, tcg_target_available_regs[ts->type],
		1483	+ s->reserved_regs);
		1484	+ /* XXX: sign extend may be needed on some targets */
		1485	+ tcg_out_movi(s, ts->type, reg, ts->val);
		1486	+ tcg_out_st(s, reg, TCG_REG_CALL_STACK, stack_offset);
		1487	+ } else {
		1488	+ tcg_abort();
		1489	+ }
		1490	+ stack_offset += sizeof(tcg_target_long);
		1491	+ }
		1492	+
		1493	+ /* assign input registers */
		1494	+ tcg_regset_set(allocated_regs, s->reserved_regs);
		1495	+ for(i = 0; i < nb_regs; i++) {
		1496	+ arg = args[nb_oargs + i];
		1497	+ ts = &s->temps[arg];
		1498	+ reg = tcg_target_call_iarg_regs[i];
		1499	+ tcg_reg_free(s, reg);
		1500	+ if (ts->val_type == TEMP_VAL_REG) {
		1501	+ if (ts->reg != reg) {
		1502	+ tcg_out_mov(s, reg, ts->reg);
		1503	+ }
		1504	+ } else if (ts->val_type == TEMP_VAL_MEM) {
		1505	+ tcg_out_ld(s, reg, ts->mem_reg, ts->mem_offset);
		1506	+ } else if (ts->val_type == TEMP_VAL_CONST) {
		1507	+ /* XXX: sign extend ? */
		1508	+ tcg_out_movi(s, ts->type, reg, ts->val);
		1509	+ } else {
		1510	+ tcg_abort();
		1511	+ }
		1512	+ tcg_regset_set_reg(allocated_regs, reg);
		1513	+ }
		1514	+
		1515	+ /* assign function address */
		1516	+ func_arg = args[nb_oargs + nb_iargs - 1];
		1517	+ arg_ct = &def->args_ct[0];
		1518	+ ts = &s->temps[func_arg];
		1519	+ const_func_arg = 0;
		1520	+ if (ts->val_type == TEMP_VAL_MEM) {
		1521	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1522	+ tcg_out_ld(s, reg, ts->mem_reg, ts->mem_offset);
		1523	+ func_arg = reg;
		1524	+ } else if (ts->val_type == TEMP_VAL_REG) {
		1525	+ reg = ts->reg;
		1526	+ if (!tcg_regset_test_reg(arg_ct->u.regs, reg)) {
		1527	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1528	+ tcg_out_mov(s, reg, ts->reg);
		1529	+ }
		1530	+ func_arg = reg;
		1531	+ } else if (ts->val_type == TEMP_VAL_CONST) {
		1532	+ if (tcg_target_const_match(ts->val, arg_ct)) {
		1533	+ const_func_arg = 1;
		1534	+ func_arg = ts->val;
		1535	+ } else {
		1536	+ reg = tcg_reg_alloc(s, arg_ct->u.regs, allocated_regs);
		1537	+ tcg_out_movi(s, ts->type, reg, ts->val);
		1538	+ func_arg = reg;
		1539	+ }
		1540	+ } else {
		1541	+ tcg_abort();
		1542	+ }
		1543	+
		1544	+ /* mark dead temporaries and free the associated registers */
		1545	+ for(i = 0; i < nb_params; i++) {
		1546	+ arg = args[nb_oargs + i];
		1547	+ if (IS_DEAD_IARG(i)) {
		1548	+ ts = &s->temps[arg];
		1549	+ if (ts->val_type != TEMP_VAL_CONST && !ts->fixed_reg) {
		1550	+ if (ts->val_type == TEMP_VAL_REG)
		1551	+ s->reg_to_temp[ts->reg] = -1;
		1552	+ ts->val_type = TEMP_VAL_DEAD;
		1553	+ }
		1554	+ }
		1555	+ }
		1556	+
		1557	+ /* clobber call registers */
		1558	+ for(reg = 0; reg < TCG_TARGET_NB_REGS; reg++) {
		1559	+ if (tcg_regset_test_reg(tcg_target_call_clobber_regs, reg)) {
		1560	+ tcg_reg_free(s, reg);
		1561	+ }
		1562	+ }
		1563	+
		1564	+ /* store globals and free associated registers (we assume the call
		1565	+ can modify any global. */
		1566	+ for(i = 0; i < s->nb_globals; i++) {
		1567	+ ts = &s->temps[i];
		1568	+ if (!ts->fixed_reg) {
		1569	+ if (ts->val_type == TEMP_VAL_REG) {
		1570	+ tcg_reg_free(s, ts->reg);
		1571	+ }
		1572	+ }
		1573	+ }
		1574	+
		1575	+ tcg_out_op(s, opc, &func_arg, &const_func_arg);
		1576	+
		1577	+ tcg_out_addi(s, TCG_REG_CALL_STACK, call_stack_size);
		1578	+
		1579	+ /* assign output registers and emit moves if needed */
		1580	+ for(i = 0; i < nb_oargs; i++) {
		1581	+ arg = args[i];
		1582	+ ts = &s->temps[arg];
		1583	+ reg = tcg_target_call_oarg_regs[i];
		1584	+ tcg_reg_free(s, reg);
		1585	+ if (ts->fixed_reg) {
		1586	+ if (ts->reg != reg) {
		1587	+ tcg_out_mov(s, ts->reg, reg);
		1588	+ }
		1589	+ } else {
		1590	+ if (ts->val_type == TEMP_VAL_REG)
		1591	+ s->reg_to_temp[ts->reg] = -1;
		1592	+ ts->val_type = TEMP_VAL_REG;
		1593	+ ts->reg = reg;
		1594	+ ts->mem_coherent = 0;
		1595	+ s->reg_to_temp[reg] = arg;
		1596	+ }
		1597	+ }
		1598	+
		1599	+ return nb_iargs + nb_oargs + def->nb_cargs + 1;
		1600	+}
		1601	+
		1602	+#ifdef CONFIG_PROFILER
		1603	+
		1604	+static int64_t dyngen_table_op_count[NB_OPS];
		1605	+
		1606	+void dump_op_count(void)
		1607	+{
		1608	+ int i;
		1609	+ FILE *f;
		1610	+ f = fopen("/tmp/op1.log", "w");
		1611	+ for(i = 0; i < INDEX_op_end; i++) {
		1612	+ fprintf(f, "%s %" PRId64 "\n", tcg_op_defs[i].name, dyngen_table_op_count[i]);
		1613	+ }
		1614	+ fclose(f);
		1615	+ f = fopen("/tmp/op2.log", "w");
		1616	+ for(i = INDEX_op_end; i < NB_OPS; i++) {
		1617	+ fprintf(f, "%s %" PRId64 "\n", tcg_op_defs[i].name, dyngen_table_op_count[i]);
		1618	+ }
		1619	+ fclose(f);
		1620	+}
		1621	+#endif
		1622	+
		1623	+
		1624	+static inline int tcg_gen_code_common(TCGContext s, uint8_t gen_code_buf,
		1625	+ int do_search_pc,
		1626	+ const uint8_t *searched_pc)
		1627	+{
		1628	+ int opc, op_index, macro_op_index;
		1629	+ const TCGOpDef *def;
		1630	+ unsigned int dead_iargs;
		1631	+ const TCGArg *args;
		1632	+
		1633	+#ifdef DEBUG_DISAS
		1634	+ if (unlikely(loglevel & CPU_LOG_TB_OP)) {
		1635	+ fprintf(logfile, "OP:\n");
		1636	+ tcg_dump_ops(s, logfile);
		1637	+ fprintf(logfile, "\n");
		1638	+ }
		1639	+#endif
		1640	+
		1641	+ tcg_liveness_analysis(s);
		1642	+
		1643	+#ifdef DEBUG_DISAS
		1644	+ if (unlikely(loglevel & CPU_LOG_TB_OP_OPT)) {
		1645	+ fprintf(logfile, "OP after la:\n");
		1646	+ tcg_dump_ops(s, logfile);
		1647	+ fprintf(logfile, "\n");
		1648	+ }
		1649	+#endif
		1650	+
		1651	+ tcg_reg_alloc_start(s);
		1652	+
		1653	+ s->code_buf = gen_code_buf;
		1654	+ s->code_ptr = gen_code_buf;
		1655	+
		1656	+ macro_op_index = -1;
		1657	+ args = gen_opparam_buf;
		1658	+ op_index = 0;
		1659	+ for(;;) {
		1660	+ opc = gen_opc_buf[op_index];
		1661	+#ifdef CONFIG_PROFILER
		1662	+ dyngen_table_op_count[opc]++;
		1663	+#endif
		1664	+ def = &tcg_op_defs[opc];
		1665	+#if 0
		1666	+ printf("%s: %d %d %d\n", def->name,
		1667	+ def->nb_oargs, def->nb_iargs, def->nb_cargs);
		1668	+ // dump_regs(s);
		1669	+#endif
		1670	+ switch(opc) {
		1671	+ case INDEX_op_mov_i32:
		1672	+#if TCG_TARGET_REG_BITS == 64
		1673	+ case INDEX_op_mov_i64:
		1674	+#endif
		1675	+ dead_iargs = s->op_dead_iargs[op_index];
		1676	+ tcg_reg_alloc_mov(s, def, args, dead_iargs);
		1677	+ break;
		1678	+ case INDEX_op_nop:
		1679	+ case INDEX_op_nop1:
		1680	+ case INDEX_op_nop2:
		1681	+ case INDEX_op_nop3:
		1682	+ break;
		1683	+ case INDEX_op_nopn:
		1684	+ args += args[0];
		1685	+ goto next;
		1686	+ case INDEX_op_macro_goto:
		1687	+ macro_op_index = op_index; /* only used for exceptions */
		1688	+ op_index = args[0] - 1;
		1689	+ args = gen_opparam_buf + args[1];
		1690	+ goto next;
		1691	+ case INDEX_op_macro_end:
		1692	+ macro_op_index = -1; /* only used for exceptions */
		1693	+ op_index = args[0] - 1;
		1694	+ args = gen_opparam_buf + args[1];
		1695	+ goto next;
		1696	+ case INDEX_op_macro_start:
		1697	+ /* must never happen here */
		1698	+ tcg_abort();
		1699	+ case INDEX_op_set_label:
		1700	+ tcg_reg_alloc_bb_end(s);
		1701	+ tcg_out_label(s, args[0], (long)s->code_ptr);
		1702	+ break;
		1703	+ case INDEX_op_call:
		1704	+ dead_iargs = s->op_dead_iargs[op_index];
		1705	+ args += tcg_reg_alloc_call(s, def, opc, args, dead_iargs);
		1706	+ goto next;
		1707	+ case INDEX_op_end:
		1708	+ goto the_end;
		1709	+ case 0 ... INDEX_op_end - 1:
		1710	+ /* legacy dyngen ops */
		1711	+#ifdef CONFIG_PROFILER
		1712	+ {
		1713	+ extern int64_t dyngen_old_op_count;
		1714	+ dyngen_old_op_count++;
		1715	+ }
		1716	+#endif
		1717	+ tcg_reg_alloc_bb_end(s);
		1718	+ if (do_search_pc) {
		1719	+ s->code_ptr += def->copy_size;
		1720	+ args += def->nb_args;
		1721	+ } else {
		1722	+ args = dyngen_op(s, opc, args);
		1723	+ }
		1724	+ goto next;
		1725	+ default:
		1726	+ /* Note: in order to speed up the code, it would be much
		1727	+ faster to have specialized register allocator functions for
		1728	+ some common argument patterns */
		1729	+ dead_iargs = s->op_dead_iargs[op_index];
		1730	+ tcg_reg_alloc_op(s, def, opc, args, dead_iargs);
		1731	+ break;
		1732	+ }
		1733	+ args += def->nb_args;
		1734	+ next: ;
		1735	+ if (do_search_pc) {
		1736	+ if (searched_pc < s->code_ptr) {
		1737	+ if (macro_op_index >= 0)
		1738	+ return macro_op_index;
		1739	+ else
		1740	+ return op_index;
		1741	+ }
		1742	+ }
		1743	+ op_index++;
		1744	+#ifndef NDEBUG
		1745	+ check_regs(s);
		1746	+#endif
		1747	+ }
		1748	+ the_end:
		1749	+ return -1;
		1750	+}
		1751	+
		1752	+int dyngen_code(TCGContext s, uint8_t gen_code_buf)
		1753	+{
		1754	+#ifdef CONFIG_PROFILER
		1755	+ {
		1756	+ extern int64_t dyngen_op_count;
		1757	+ extern int dyngen_op_count_max;
		1758	+ int n;
		1759	+ n = (gen_opc_ptr - gen_opc_buf);
		1760	+ dyngen_op_count += n;
		1761	+ if (n > dyngen_op_count_max)
		1762	+ dyngen_op_count_max = n;
		1763	+ }
		1764	+#endif
		1765	+
		1766	+ tcg_gen_code_common(s, gen_code_buf, 0, NULL);
		1767	+
		1768	+ /* flush instruction cache */
		1769	+ flush_icache_range((unsigned long)gen_code_buf,
		1770	+ (unsigned long)s->code_ptr);
		1771	+ return s->code_ptr - gen_code_buf;
		1772	+}
		1773	+
		1774	+/* return the index of the micro operation such as the pc after is <
		1775	+ search_pc. Note: gen_code_buf is accessed during the operation, but
		1776	+ its content should not be modified. Return -1 if not found. */
		1777	+int dyngen_code_search_pc(TCGContext s, uint8_t gen_code_buf,
		1778	+ const uint8_t *searched_pc)
		1779	+{
		1780	+ return tcg_gen_code_common(s, gen_code_buf, 1, searched_pc);
		1781	+}

tcg/tcg.h 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#include "tcg-target.h"
		25	+
		26	+#if TCG_TARGET_REG_BITS == 32
		27	+typedef int32_t tcg_target_long;
		28	+typedef uint32_t tcg_target_ulong;
		29	+#define TCG_PRIlx PRIx32
		30	+#define TCG_PRIld PRId32
		31	+#elif TCG_TARGET_REG_BITS == 64
		32	+typedef int64_t tcg_target_long;
		33	+typedef uint64_t tcg_target_ulong;
		34	+#define TCG_PRIlx PRIx64
		35	+#define TCG_PRIld PRId64
		36	+#else
		37	+#error unsupported
		38	+#endif
		39	+
		40	+#if TCG_TARGET_NB_REGS <= 32
		41	+typedef uint32_t TCGRegSet;
		42	+#elif TCG_TARGET_NB_REGS <= 64
		43	+typedef uint64_t TCGRegSet;
		44	+#else
		45	+#error unsupported
		46	+#endif
		47	+
		48	+enum {
		49	+#define DEF(s, n, copy_size) INDEX_op_ ## s,
		50	+#include "tcg-opc.h"
		51	+#undef DEF
		52	+ NB_OPS,
		53	+};
		54	+
		55	+#define tcg_regset_clear(d) (d) = 0
		56	+#define tcg_regset_set(d, s) (d) = (s)
		57	+#define tcg_regset_set32(d, reg, val32) (d) \|= (val32) << (reg)
		58	+#define tcg_regset_set_reg(d, r) (d) \|= 1 << (r)
		59	+#define tcg_regset_reset_reg(d, r) (d) &= ~(1 << (r))
		60	+#define tcg_regset_test_reg(d, r) (((d) >> (r)) & 1)
		61	+#define tcg_regset_or(d, a, b) (d) = (a) \| (b)
		62	+#define tcg_regset_and(d, a, b) (d) = (a) & (b)
		63	+#define tcg_regset_andnot(d, a, b) (d) = (a) & ~(b)
		64	+#define tcg_regset_not(d, a) (d) = ~(a)
		65	+
		66	+typedef struct TCGRelocation {
		67	+ struct TCGRelocation *next;
		68	+ int type;
		69	+ uint8_t *ptr;
		70	+ tcg_target_long addend;
		71	+} TCGRelocation;
		72	+
		73	+typedef struct TCGLabel {
		74	+ int has_value;
		75	+ union {
		76	+ tcg_target_ulong value;
		77	+ TCGRelocation *first_reloc;
		78	+ } u;
		79	+} TCGLabel;
		80	+
		81	+typedef struct TCGPool {
		82	+ struct TCGPool *next;
		83	+ int size;
		84	+ uint8_t data[0];
		85	+} TCGPool;
		86	+
		87	+#define TCG_POOL_CHUNK_SIZE 32768
		88	+
		89	+#define TCG_MAX_LABELS 512
		90	+
		91	+#define TCG_MAX_TEMPS 256
		92	+
		93	+typedef int TCGType;
		94	+
		95	+#define TCG_TYPE_I32 0
		96	+#define TCG_TYPE_I64 1
		97	+
		98	+#if TCG_TARGET_REG_BITS == 32
		99	+#define TCG_TYPE_PTR TCG_TYPE_I32
		100	+#else
		101	+#define TCG_TYPE_PTR TCG_TYPE_I64
		102	+#endif
		103	+
		104	+typedef tcg_target_ulong TCGArg;
		105	+
		106	+/* call flags */
		107	+#define TCG_CALL_TYPE_MASK 0x000f
		108	+#define TCG_CALL_TYPE_STD 0x0000 /* standard C call */
		109	+#define TCG_CALL_TYPE_REGPARM_1 0x0001 /* i386 style regparm call (1 reg) */
		110	+#define TCG_CALL_TYPE_REGPARM_2 0x0002 /* i386 style regparm call (2 regs) */
		111	+#define TCG_CALL_TYPE_REGPARM 0x0003 /* i386 style regparm call (3 regs) */
		112	+
		113	+typedef enum {
		114	+ TCG_COND_EQ,
		115	+ TCG_COND_NE,
		116	+ TCG_COND_LT,
		117	+ TCG_COND_GE,
		118	+ TCG_COND_LE,
		119	+ TCG_COND_GT,
		120	+ /* unsigned */
		121	+ TCG_COND_LTU,
		122	+ TCG_COND_GEU,
		123	+ TCG_COND_LEU,
		124	+ TCG_COND_GTU,
		125	+} TCGCond;
		126	+
		127	+#define TEMP_VAL_DEAD 0
		128	+#define TEMP_VAL_REG 1
		129	+#define TEMP_VAL_MEM 2
		130	+#define TEMP_VAL_CONST 3
		131	+
		132	+/* XXX: optimize memory layout */
		133	+typedef struct TCGTemp {
		134	+ TCGType base_type;
		135	+ TCGType type;
		136	+ int val_type;
		137	+ int reg;
		138	+ tcg_target_long val;
		139	+ int mem_reg;
		140	+ tcg_target_long mem_offset;
		141	+ unsigned int fixed_reg:1;
		142	+ unsigned int mem_coherent:1;
		143	+ unsigned int mem_allocated:1;
		144	+ const char *name;
		145	+} TCGTemp;
		146	+
		147	+typedef struct TCGHelperInfo {
		148	+ void *func;
		149	+ const char *name;
		150	+} TCGHelperInfo;
		151	+
		152	+typedef struct TCGContext TCGContext;
		153	+
		154	+typedef void TCGMacroFunc(TCGContext s, int macro_id, const int dead_args);
		155	+
		156	+struct TCGContext {
		157	+ uint8_t pool_cur, pool_end;
		158	+ TCGPool pool_first, pool_current;
		159	+ TCGLabel *labels;
		160	+ int nb_labels;
		161	+ TCGTemp temps; / globals first, temps after */
		162	+ int nb_globals;
		163	+ int nb_temps;
		164	+ /* constant indexes (end of temp array) */
		165	+ int const_start;
		166	+ int const_end;
		167	+
		168	+ /* goto_tb support */
		169	+ uint8_t *code_buf;
		170	+ unsigned long *tb_next;
		171	+ uint16_t *tb_next_offset;
		172	+ uint16_t tb_jmp_offset; / != NULL if USE_DIRECT_JUMP */
		173	+
		174	+ uint16_t op_dead_iargs; / for each operation, each bit tells if the
		175	+ corresponding input argument is dead */
		176	+ /* tells in which temporary a given register is. It does not take
		177	+ into account fixed registers */
		178	+ int reg_to_temp[TCG_TARGET_NB_REGS];
		179	+ TCGRegSet reserved_regs;
		180	+ tcg_target_long current_frame_offset;
		181	+ tcg_target_long frame_start;
		182	+ tcg_target_long frame_end;
		183	+ int frame_reg;
		184	+
		185	+ uint8_t *code_ptr;
		186	+ TCGTemp static_temps[TCG_MAX_TEMPS];
		187	+
		188	+ TCGMacroFunc *macro_func;
		189	+ TCGHelperInfo *helpers;
		190	+ int nb_helpers;
		191	+ int allocated_helpers;
		192	+};
		193	+
		194	+extern TCGContext tcg_ctx;
		195	+extern uint16_t *gen_opc_ptr;
		196	+extern TCGArg *gen_opparam_ptr;
		197	+extern uint16_t gen_opc_buf[];
		198	+extern TCGArg gen_opparam_buf[];
		199	+
		200	+/* pool based memory allocation */
		201	+
		202	+void tcg_malloc_internal(TCGContext s, int size);
		203	+void tcg_pool_reset(TCGContext *s);
		204	+void tcg_pool_delete(TCGContext *s);
		205	+
		206	+static inline void *tcg_malloc(int size)
		207	+{
		208	+ TCGContext *s = &tcg_ctx;
		209	+ uint8_t ptr, ptr_end;
		210	+ size = (size + sizeof(long) - 1) & ~(sizeof(long) - 1);
		211	+ ptr = s->pool_cur;
		212	+ ptr_end = ptr + size;
		213	+ if (unlikely(ptr_end > s->pool_end)) {
		214	+ return tcg_malloc_internal(&tcg_ctx, size);
		215	+ } else {
		216	+ s->pool_cur = ptr_end;
		217	+ return ptr;
		218	+ }
		219	+}
		220	+
		221	+void tcg_context_init(TCGContext *s);
		222	+void tcg_func_start(TCGContext *s);
		223	+
		224	+int dyngen_code(TCGContext s, uint8_t gen_code_buf);
		225	+int dyngen_code_search_pc(TCGContext s, uint8_t gen_code_buf,
		226	+ const uint8_t *searched_pc);
		227	+
		228	+void tcg_set_frame(TCGContext *s, int reg,
		229	+ tcg_target_long start, tcg_target_long size);
		230	+void tcg_set_macro_func(TCGContext s, TCGMacroFunc func);
		231	+int tcg_global_reg_new(TCGType type, int reg, const char *name);
		232	+int tcg_global_mem_new(TCGType type, int reg, tcg_target_long offset,
		233	+ const char *name);
		234	+int tcg_temp_new(TCGType type);
		235	+char tcg_get_arg_str(TCGContext s, char *buf, int buf_size, TCGArg arg);
		236	+
		237	+#define TCG_CT_ALIAS 0x80
		238	+#define TCG_CT_IALIAS 0x40
		239	+#define TCG_CT_REG 0x01
		240	+#define TCG_CT_CONST 0x02 /* any constant of register size */
		241	+
		242	+typedef struct TCGArgConstraint {
		243	+ uint32_t ct;
		244	+ union {
		245	+ TCGRegSet regs;
		246	+ } u;
		247	+} TCGArgConstraint;
		248	+
		249	+#define TCG_MAX_OP_ARGS 16
		250	+
		251	+#define TCG_OPF_BB_END 0x01 /* instruction defines the end of a basic
		252	+ block */
		253	+#define TCG_OPF_CALL_CLOBBER 0x02 /* instruction clobbers call registers */
		254	+
		255	+typedef struct TCGOpDef {
		256	+ const char *name;
		257	+ uint8_t nb_oargs, nb_iargs, nb_cargs, nb_args;
		258	+ uint8_t flags;
		259	+ uint16_t copy_size;
		260	+ TCGArgConstraint *args_ct;
		261	+ int *sorted_args;
		262	+} TCGOpDef;
		263	+
		264	+typedef struct TCGTargetOpDef {
		265	+ int op;
		266	+ const char *args_ct_str[TCG_MAX_OP_ARGS];
		267	+} TCGTargetOpDef;
		268	+
		269	+extern TCGOpDef tcg_op_defs[];
		270	+
		271	+void tcg_target_init(TCGContext *s);
		272	+
		273	+#define tcg_abort() \
		274	+do {\
		275	+ fprintf(stderr, "%s:%d: tcg fatal error\n", __FILE__, __LINE__);\
		276	+ abort();\
		277	+} while (0)
		278	+
		279	+void tcg_add_target_add_op_defs(const TCGTargetOpDef *tdefs);
		280	+
		281	+void tcg_gen_call(TCGContext *s, TCGArg func, unsigned int flags,
		282	+ unsigned int nb_rets, const TCGArg *rets,
		283	+ unsigned int nb_params, const TCGArg *args1);
		284	+void tcg_gen_shifti_i64(TCGArg ret, TCGArg arg1,
		285	+ int c, int right, int arith);
		286	+
		287	+/* only used for debugging purposes */
		288	+void tcg_register_helper(void func, const char name);
		289	+#define TCG_HELPER(func) tcg_register_helper(func, #func)
		290	+const char tcg_helper_get_name(TCGContext s, void *func);
		291	+void tcg_dump_ops(TCGContext s, FILE outfile);
		292	+
		293	+void dump_ops(const uint16_t opc_buf, const TCGArg opparam_buf);
		294	+int tcg_const_i32(int32_t val);
		295	+int tcg_const_i64(int64_t val);
		296	+
		297	+#if TCG_TARGET_REG_BITS == 32
		298	+#define tcg_const_ptr tcg_const_i32
		299	+#define tcg_add_ptr tcg_add_i32
		300	+#define tcg_sub_ptr tcg_sub_i32
		301	+#else
		302	+#define tcg_const_ptr tcg_const_i64
		303	+#define tcg_add_ptr tcg_add_i64
		304	+#define tcg_sub_ptr tcg_sub_i64
		305	+#endif
		306	+
		307	+void tcg_out_reloc(TCGContext s, uint8_t code_ptr, int type,
		308	+ int label_index, long addend);
		309	+void tcg_reg_alloc_start(TCGContext *s);
		310	+void tcg_reg_alloc_bb_end(TCGContext *s);
		311	+void tcg_liveness_analysis(TCGContext *s);
		312	+const TCGArg tcg_gen_code_op(TCGContext s, int opc, const TCGArg *args1,
		313	+ unsigned int dead_iargs);
		314	+
		315	+const TCGArg dyngen_op(TCGContext s, int opc, const TCGArg *opparam_ptr);
		316	+
		317	+/* tcg-runtime.c */
		318	+int64_t tcg_helper_shl_i64(int64_t arg1, int64_t arg2);
		319	+int64_t tcg_helper_shr_i64(int64_t arg1, int64_t arg2);
		320	+int64_t tcg_helper_sar_i64(int64_t arg1, int64_t arg2);
		321	+int64_t tcg_helper_div_i64(int64_t arg1, int64_t arg2);
		322	+int64_t tcg_helper_rem_i64(int64_t arg1, int64_t arg2);
		323	+uint64_t tcg_helper_divu_i64(uint64_t arg1, uint64_t arg2);
		324	+uint64_t tcg_helper_remu_i64(uint64_t arg1, uint64_t arg2);

tcg/x86_64/tcg-target.c 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+const char *tcg_target_reg_names[TCG_TARGET_NB_REGS] = {
		25	+ "%rax",
		26	+ "%rcx",
		27	+ "%rdx",
		28	+ "%rbx",
		29	+ "%rsp",
		30	+ "%rbp",
		31	+ "%rsi",
		32	+ "%rdi",
		33	+ "%r8",
		34	+ "%r9",
		35	+ "%r10",
		36	+ "%r11",
		37	+ "%r12",
		38	+ "%r13",
		39	+ "%r14",
		40	+ "%r15",
		41	+};
		42	+
		43	+int tcg_target_reg_alloc_order[TCG_TARGET_NB_REGS] = {
		44	+ TCG_REG_RDI,
		45	+ TCG_REG_RSI,
		46	+ TCG_REG_RDX,
		47	+ TCG_REG_RCX,
		48	+ TCG_REG_R8,
		49	+ TCG_REG_R9,
		50	+ TCG_REG_RAX,
		51	+ TCG_REG_R10,
		52	+ TCG_REG_R11,
		53	+
		54	+ TCG_REG_RBP,
		55	+ TCG_REG_RBX,
		56	+ TCG_REG_R12,
		57	+ TCG_REG_R13,
		58	+ TCG_REG_R14,
		59	+ TCG_REG_R15,
		60	+};
		61	+
		62	+const int tcg_target_call_iarg_regs[6] = {
		63	+ TCG_REG_RDI,
		64	+ TCG_REG_RSI,
		65	+ TCG_REG_RDX,
		66	+ TCG_REG_RCX,
		67	+ TCG_REG_R8,
		68	+ TCG_REG_R9,
		69	+};
		70	+
		71	+const int tcg_target_call_oarg_regs[2] = {
		72	+ TCG_REG_RAX,
		73	+ TCG_REG_RDX
		74	+};
		75	+
		76	+static void patch_reloc(uint8_t *code_ptr, int type,
		77	+ tcg_target_long value)
		78	+{
		79	+ switch(type) {
		80	+ case R_X86_64_32:
		81	+ if (value != (uint32_t)value)
		82	+ tcg_abort();
		83	+ (uint32_t )code_ptr = value;
		84	+ break;
		85	+ case R_X86_64_32S:
		86	+ if (value != (int32_t)value)
		87	+ tcg_abort();
		88	+ (uint32_t )code_ptr = value;
		89	+ break;
		90	+ case R_386_PC32:
		91	+ value -= (long)code_ptr;
		92	+ if (value != (int32_t)value)
		93	+ tcg_abort();
		94	+ (uint32_t )code_ptr = value;
		95	+ break;
		96	+ default:
		97	+ tcg_abort();
		98	+ }
		99	+}
		100	+
		101	+/* maximum number of register used for input function arguments */
		102	+static inline int tcg_target_get_call_iarg_regs_count(int flags)
		103	+{
		104	+ return 6;
		105	+}
		106	+
		107	+/* parse target specific constraints */
		108	+int target_parse_constraint(TCGArgConstraint ct, const char *pct_str)
		109	+{
		110	+ const char *ct_str;
		111	+
		112	+ ct_str = *pct_str;
		113	+ switch(ct_str[0]) {
		114	+ case 'a':
		115	+ ct->ct \|= TCG_CT_REG;
		116	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_RAX);
		117	+ break;
		118	+ case 'b':
		119	+ ct->ct \|= TCG_CT_REG;
		120	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_RBX);
		121	+ break;
		122	+ case 'c':
		123	+ ct->ct \|= TCG_CT_REG;
		124	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_RCX);
		125	+ break;
		126	+ case 'd':
		127	+ ct->ct \|= TCG_CT_REG;
		128	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_RDX);
		129	+ break;
		130	+ case 'S':
		131	+ ct->ct \|= TCG_CT_REG;
		132	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_RSI);
		133	+ break;
		134	+ case 'D':
		135	+ ct->ct \|= TCG_CT_REG;
		136	+ tcg_regset_set_reg(ct->u.regs, TCG_REG_RDI);
		137	+ break;
		138	+ case 'q':
		139	+ ct->ct \|= TCG_CT_REG;
		140	+ tcg_regset_set32(ct->u.regs, 0, 0xf);
		141	+ break;
		142	+ case 'r':
		143	+ ct->ct \|= TCG_CT_REG;
		144	+ tcg_regset_set32(ct->u.regs, 0, 0xffff);
		145	+ break;
		146	+ case 'L': /* qemu_ld/st constraint */
		147	+ ct->ct \|= TCG_CT_REG;
		148	+ tcg_regset_set32(ct->u.regs, 0, 0xffff);
		149	+ tcg_regset_reset_reg(ct->u.regs, TCG_REG_RSI);
		150	+ tcg_regset_reset_reg(ct->u.regs, TCG_REG_RDI);
		151	+ break;
		152	+ case 'e':
		153	+ ct->ct \|= TCG_CT_CONST_S32;
		154	+ break;
		155	+ case 'Z':
		156	+ ct->ct \|= TCG_CT_CONST_U32;
		157	+ break;
		158	+ default:
		159	+ return -1;
		160	+ }
		161	+ ct_str++;
		162	+ *pct_str = ct_str;
		163	+ return 0;
		164	+}
		165	+
		166	+/* test if a constant matches the constraint */
		167	+static inline int tcg_target_const_match(tcg_target_long val,
		168	+ const TCGArgConstraint *arg_ct)
		169	+{
		170	+ int ct;
		171	+ ct = arg_ct->ct;
		172	+ if (ct & TCG_CT_CONST)
		173	+ return 1;
		174	+ else if ((ct & TCG_CT_CONST_S32) && val == (int32_t)val)
		175	+ return 1;
		176	+ else if ((ct & TCG_CT_CONST_U32) && val == (uint32_t)val)
		177	+ return 1;
		178	+ else
		179	+ return 0;
		180	+}
		181	+
		182	+#define ARITH_ADD 0
		183	+#define ARITH_OR 1
		184	+#define ARITH_ADC 2
		185	+#define ARITH_SBB 3
		186	+#define ARITH_AND 4
		187	+#define ARITH_SUB 5
		188	+#define ARITH_XOR 6
		189	+#define ARITH_CMP 7
		190	+
		191	+#define SHIFT_SHL 4
		192	+#define SHIFT_SHR 5
		193	+#define SHIFT_SAR 7
		194	+
		195	+#define JCC_JMP (-1)
		196	+#define JCC_JO 0x0
		197	+#define JCC_JNO 0x1
		198	+#define JCC_JB 0x2
		199	+#define JCC_JAE 0x3
		200	+#define JCC_JE 0x4
		201	+#define JCC_JNE 0x5
		202	+#define JCC_JBE 0x6
		203	+#define JCC_JA 0x7
		204	+#define JCC_JS 0x8
		205	+#define JCC_JNS 0x9
		206	+#define JCC_JP 0xa
		207	+#define JCC_JNP 0xb
		208	+#define JCC_JL 0xc
		209	+#define JCC_JGE 0xd
		210	+#define JCC_JLE 0xe
		211	+#define JCC_JG 0xf
		212	+
		213	+#define P_EXT 0x100 /* 0x0f opcode prefix */
		214	+#define P_REXW 0x200 /* set rex.w = 1 */
		215	+#define P_REX 0x400 /* force rex usage */
		216	+
		217	+static const uint8_t tcg_cond_to_jcc[10] = {
		218	+ [TCG_COND_EQ] = JCC_JE,
		219	+ [TCG_COND_NE] = JCC_JNE,
		220	+ [TCG_COND_LT] = JCC_JL,
		221	+ [TCG_COND_GE] = JCC_JGE,
		222	+ [TCG_COND_LE] = JCC_JLE,
		223	+ [TCG_COND_GT] = JCC_JG,
		224	+ [TCG_COND_LTU] = JCC_JB,
		225	+ [TCG_COND_GEU] = JCC_JAE,
		226	+ [TCG_COND_LEU] = JCC_JBE,
		227	+ [TCG_COND_GTU] = JCC_JA,
		228	+};
		229	+
		230	+static inline void tcg_out_opc(TCGContext *s, int opc, int r, int rm, int x)
		231	+{
		232	+ int rex;
		233	+ rex = ((opc >> 6) & 0x8) \| ((r >> 1) & 0x4) \|
		234	+ ((x >> 2) & 2) \| ((rm >> 3) & 1);
		235	+ if (rex \|\| (opc & P_REX)) {
		236	+ tcg_out8(s, rex \| 0x40);
		237	+ }
		238	+ if (opc & P_EXT)
		239	+ tcg_out8(s, 0x0f);
		240	+ tcg_out8(s, opc);
		241	+}
		242	+
		243	+static inline void tcg_out_modrm(TCGContext *s, int opc, int r, int rm)
		244	+{
		245	+ tcg_out_opc(s, opc, r, rm, 0);
		246	+ tcg_out8(s, 0xc0 \| ((r & 7) << 3) \| (rm & 7));
		247	+}
		248	+
		249	+/* rm < 0 means no register index plus (-rm - 1 immediate bytes) */
		250	+static inline void tcg_out_modrm_offset(TCGContext *s, int opc, int r, int rm,
		251	+ tcg_target_long offset)
		252	+{
		253	+ if (rm < 0) {
		254	+ tcg_target_long val;
		255	+ tcg_out_opc(s, opc, r, 0, 0);
		256	+ val = offset - ((tcg_target_long)s->code_ptr + 5 + (-rm - 1));
		257	+ if (val == (int32_t)val) {
		258	+ /* eip relative */
		259	+ tcg_out8(s, 0x05 \| ((r & 7) << 3));
		260	+ tcg_out32(s, val);
		261	+ } else if (offset == (int32_t)offset) {
		262	+ tcg_out8(s, 0x04 \| ((r & 7) << 3));
		263	+ tcg_out8(s, 0x25); /* sib */
		264	+ tcg_out32(s, offset);
		265	+ } else {
		266	+ tcg_abort();
		267	+ }
		268	+ } else if (offset == 0 && (rm & 7) != TCG_REG_RBP) {
		269	+ tcg_out_opc(s, opc, r, rm, 0);
		270	+ if ((rm & 7) == TCG_REG_RSP) {
		271	+ tcg_out8(s, 0x04 \| ((r & 7) << 3));
		272	+ tcg_out8(s, 0x24);
		273	+ } else {
		274	+ tcg_out8(s, 0x00 \| ((r & 7) << 3) \| (rm & 7));
		275	+ }
		276	+ } else if ((int8_t)offset == offset) {
		277	+ tcg_out_opc(s, opc, r, rm, 0);
		278	+ if ((rm & 7) == TCG_REG_RSP) {
		279	+ tcg_out8(s, 0x44 \| ((r & 7) << 3));
		280	+ tcg_out8(s, 0x24);
		281	+ } else {
		282	+ tcg_out8(s, 0x40 \| ((r & 7) << 3) \| (rm & 7));
		283	+ }
		284	+ tcg_out8(s, offset);
		285	+ } else {
		286	+ tcg_out_opc(s, opc, r, rm, 0);
		287	+ if ((rm & 7) == TCG_REG_RSP) {
		288	+ tcg_out8(s, 0x84 \| ((r & 7) << 3));
		289	+ tcg_out8(s, 0x24);
		290	+ } else {
		291	+ tcg_out8(s, 0x80 \| ((r & 7) << 3) \| (rm & 7));
		292	+ }
		293	+ tcg_out32(s, offset);
		294	+ }
		295	+}
		296	+
		297	+/* XXX: incomplete. index must be different from ESP */
		298	+static void tcg_out_modrm_offset2(TCGContext *s, int opc, int r, int rm,
		299	+ int index, int shift,
		300	+ tcg_target_long offset)
		301	+{
		302	+ int mod;
		303	+ if (rm == -1)
		304	+ tcg_abort();
		305	+ if (offset == 0 && (rm & 7) != TCG_REG_RBP) {
		306	+ mod = 0;
		307	+ } else if (offset == (int8_t)offset) {
		308	+ mod = 0x40;
		309	+ } else if (offset == (int32_t)offset) {
		310	+ mod = 0x80;
		311	+ } else {
		312	+ tcg_abort();
		313	+ }
		314	+ if (index == -1) {
		315	+ tcg_out_opc(s, opc, r, rm, 0);
		316	+ if ((rm & 7) == TCG_REG_RSP) {
		317	+ tcg_out8(s, mod \| ((r & 7) << 3) \| 0x04);
		318	+ tcg_out8(s, 0x04 \| (rm & 7));
		319	+ } else {
		320	+ tcg_out8(s, mod \| ((r & 7) << 3) \| (rm & 7));
		321	+ }
		322	+ } else {
		323	+ tcg_out_opc(s, opc, r, rm, index);
		324	+ tcg_out8(s, mod \| ((r & 7) << 3) \| 0x04);
		325	+ tcg_out8(s, (shift << 6) \| ((index & 7) << 3) \| (rm & 7));
		326	+ }
		327	+ if (mod == 0x40) {
		328	+ tcg_out8(s, offset);
		329	+ } else if (mod == 0x80) {
		330	+ tcg_out32(s, offset);
		331	+ }
		332	+}
		333	+
		334	+static inline void tcg_out_mov(TCGContext *s, int ret, int arg)
		335	+{
		336	+ tcg_out_modrm(s, 0x8b \| P_REXW, ret, arg);
		337	+}
		338	+
		339	+static inline void tcg_out_movi(TCGContext *s, TCGType type,
		340	+ int ret, tcg_target_long arg)
		341	+{
		342	+ if (arg == 0) {
		343	+ tcg_out_modrm(s, 0x01 \| (ARITH_XOR << 3), ret, ret); /* xor r0,r0 */
		344	+ } else if (arg == (uint32_t)arg \|\| type == TCG_TYPE_I32) {
		345	+ tcg_out_opc(s, 0xb8 + (ret & 7), 0, ret, 0);
		346	+ tcg_out32(s, arg);
		347	+ } else if (arg == (int32_t)arg) {
		348	+ tcg_out_modrm(s, 0xc7 \| P_REXW, 0, ret);
		349	+ tcg_out32(s, arg);
		350	+ } else {
		351	+ tcg_out_opc(s, (0xb8 + (ret & 7)) \| P_REXW, 0, ret, 0);
		352	+ tcg_out32(s, arg);
		353	+ tcg_out32(s, arg >> 32);
		354	+ }
		355	+}
		356	+
		357	+static inline void tcg_out_ld(TCGContext *s, int ret,
		358	+ int arg1, tcg_target_long arg2)
		359	+{
		360	+ tcg_out_modrm_offset(s, 0x8b \| P_REXW, ret, arg1, arg2); /* movq */
		361	+}
		362	+
		363	+static inline void tcg_out_st(TCGContext *s, int arg,
		364	+ int arg1, tcg_target_long arg2)
		365	+{
		366	+ tcg_out_modrm_offset(s, 0x89 \| P_REXW, arg, arg1, arg2); /* movq */
		367	+}
		368	+
		369	+static inline void tgen_arithi32(TCGContext *s, int c, int r0, int32_t val)
		370	+{
		371	+ if (val == (int8_t)val) {
		372	+ tcg_out_modrm(s, 0x83, c, r0);
		373	+ tcg_out8(s, val);
		374	+ } else {
		375	+ tcg_out_modrm(s, 0x81, c, r0);
		376	+ tcg_out32(s, val);
		377	+ }
		378	+}
		379	+
		380	+static inline void tgen_arithi64(TCGContext *s, int c, int r0, int64_t val)
		381	+{
		382	+ if (val == (int8_t)val) {
		383	+ tcg_out_modrm(s, 0x83 \| P_REXW, c, r0);
		384	+ tcg_out8(s, val);
		385	+ } else if (val == (int32_t)val) {
		386	+ tcg_out_modrm(s, 0x81 \| P_REXW, c, r0);
		387	+ tcg_out32(s, val);
		388	+ } else if (c == ARITH_AND && val == (uint32_t)val) {
		389	+ tcg_out_modrm(s, 0x81, c, r0);
		390	+ tcg_out32(s, val);
		391	+ } else {
		392	+ tcg_abort();
		393	+ }
		394	+}
		395	+
		396	+void tcg_out_addi(TCGContext *s, int reg, tcg_target_long val)
		397	+{
		398	+ if (val != 0)
		399	+ tgen_arithi64(s, ARITH_ADD, reg, val);
		400	+}
		401	+
		402	+static void tcg_out_jxx(TCGContext *s, int opc, int label_index)
		403	+{
		404	+ int32_t val, val1;
		405	+ TCGLabel *l = &s->labels[label_index];
		406	+
		407	+ if (l->has_value) {
		408	+ val = l->u.value - (tcg_target_long)s->code_ptr;
		409	+ val1 = val - 2;
		410	+ if ((int8_t)val1 == val1) {
		411	+ if (opc == -1)
		412	+ tcg_out8(s, 0xeb);
		413	+ else
		414	+ tcg_out8(s, 0x70 + opc);
		415	+ tcg_out8(s, val1);
		416	+ } else {
		417	+ if (opc == -1) {
		418	+ tcg_out8(s, 0xe9);
		419	+ tcg_out32(s, val - 5);
		420	+ } else {
		421	+ tcg_out8(s, 0x0f);
		422	+ tcg_out8(s, 0x80 + opc);
		423	+ tcg_out32(s, val - 6);
		424	+ }
		425	+ }
		426	+ } else {
		427	+ if (opc == -1) {
		428	+ tcg_out8(s, 0xe9);
		429	+ } else {
		430	+ tcg_out8(s, 0x0f);
		431	+ tcg_out8(s, 0x80 + opc);
		432	+ }
		433	+ tcg_out_reloc(s, s->code_ptr, R_386_PC32, label_index, -4);
		434	+ tcg_out32(s, -4);
		435	+ }
		436	+}
		437	+
		438	+static void tcg_out_brcond(TCGContext *s, int cond,
		439	+ TCGArg arg1, TCGArg arg2, int const_arg2,
		440	+ int label_index, int rexw)
		441	+{
		442	+ int c;
		443	+ if (const_arg2) {
		444	+ if (arg2 == 0) {
		445	+ /* use test */
		446	+ switch(cond) {
		447	+ case TCG_COND_EQ:
		448	+ c = JCC_JNE;
		449	+ break;
		450	+ case TCG_COND_NE:
		451	+ c = JCC_JNE;
		452	+ break;
		453	+ case TCG_COND_LT:
		454	+ c = JCC_JS;
		455	+ break;
		456	+ case TCG_COND_GE:
		457	+ c = JCC_JNS;
		458	+ break;
		459	+ default:
		460	+ goto do_cmpi;
		461	+ }
		462	+ /* test r, r */
		463	+ tcg_out_modrm(s, 0x85 \| rexw, arg1, arg1);
		464	+ tcg_out_jxx(s, c, label_index);
		465	+ } else {
		466	+ do_cmpi:
		467	+ if (rexw)
		468	+ tgen_arithi64(s, ARITH_CMP, arg1, arg2);
		469	+ else
		470	+ tgen_arithi32(s, ARITH_CMP, arg1, arg2);
		471	+ tcg_out_jxx(s, tcg_cond_to_jcc[cond], label_index);
		472	+ }
		473	+ } else {
		474	+ tcg_out_modrm(s, 0x01 \| (ARITH_CMP << 3) \| rexw, arg1, arg2);
		475	+ tcg_out_jxx(s, tcg_cond_to_jcc[cond], label_index);
		476	+ }
		477	+}
		478	+
		479	+#if defined(CONFIG_SOFTMMU)
		480	+extern void __ldb_mmu(void);
		481	+extern void __ldw_mmu(void);
		482	+extern void __ldl_mmu(void);
		483	+extern void __ldq_mmu(void);
		484	+
		485	+extern void __stb_mmu(void);
		486	+extern void __stw_mmu(void);
		487	+extern void __stl_mmu(void);
		488	+extern void __stq_mmu(void);
		489	+
		490	+
		491	+static void *qemu_ld_helpers[4] = {
		492	+ __ldb_mmu,
		493	+ __ldw_mmu,
		494	+ __ldl_mmu,
		495	+ __ldq_mmu,
		496	+};
		497	+
		498	+static void *qemu_st_helpers[4] = {
		499	+ __stb_mmu,
		500	+ __stw_mmu,
		501	+ __stl_mmu,
		502	+ __stq_mmu,
		503	+};
		504	+#endif
		505	+
		506	+static void tcg_out_qemu_ld(TCGContext s, const TCGArg args,
		507	+ int opc)
		508	+{
		509	+ int addr_reg, data_reg, r0, r1, mem_index, s_bits, bswap, rexw;
		510	+#if defined(CONFIG_SOFTMMU)
		511	+ uint8_t label1_ptr, label2_ptr;
		512	+#endif
		513	+
		514	+ data_reg = *args++;
		515	+ addr_reg = *args++;
		516	+ mem_index = *args;
		517	+ s_bits = opc & 3;
		518	+
		519	+ r0 = TCG_REG_RDI;
		520	+ r1 = TCG_REG_RSI;
		521	+
		522	+#if TARGET_LONG_BITS == 32
		523	+ rexw = 0;
		524	+#else
		525	+ rexw = P_REXW;
		526	+#endif
		527	+#if defined(CONFIG_SOFTMMU)
		528	+ /* mov */
		529	+ tcg_out_modrm(s, 0x8b \| rexw, r1, addr_reg);
		530	+
		531	+ /* mov */
		532	+ tcg_out_modrm(s, 0x8b \| rexw, r0, addr_reg);
		533	+
		534	+ tcg_out_modrm(s, 0xc1 \| rexw, 5, r1); /* shr $x, r1 */
		535	+ tcg_out8(s, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS);
		536	+
		537	+ tcg_out_modrm(s, 0x81 \| rexw, 4, r0); /* andl $x, r0 */
		538	+ tcg_out32(s, TARGET_PAGE_MASK \| ((1 << s_bits) - 1));
		539	+
		540	+ tcg_out_modrm(s, 0x81, 4, r1); /* andl $x, r1 */
		541	+ tcg_out32(s, (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS);
		542	+
		543	+ /* lea offset(r1, env), r1 */
		544	+ tcg_out_modrm_offset2(s, 0x8d \| P_REXW, r1, r1, TCG_AREG0, 0,
		545	+ offsetof(CPUState, tlb_table[mem_index][0].addr_read));
		546	+
		547	+ /* cmp 0(r1), r0 */
		548	+ tcg_out_modrm_offset(s, 0x3b \| rexw, r0, r1, 0);
		549	+
		550	+ /* mov */
		551	+ tcg_out_modrm(s, 0x8b \| rexw, r0, addr_reg);
		552	+
		553	+ /* je label1 */
		554	+ tcg_out8(s, 0x70 + JCC_JE);
		555	+ label1_ptr = s->code_ptr;
		556	+ s->code_ptr++;
		557	+
		558	+ /* XXX: move that code at the end of the TB */
		559	+ tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_RSI, mem_index);
		560	+ tcg_out8(s, 0xe8);
		561	+ tcg_out32(s, (tcg_target_long)qemu_ld_helpers[s_bits] -
		562	+ (tcg_target_long)s->code_ptr - 4);
		563	+
		564	+ switch(opc) {
		565	+ case 0 \| 4:
		566	+ /* movsbq */
		567	+ tcg_out_modrm(s, 0xbe \| P_EXT \| P_REXW, data_reg, TCG_REG_RAX);
		568	+ break;
		569	+ case 1 \| 4:
		570	+ /* movswq */
		571	+ tcg_out_modrm(s, 0xbf \| P_EXT \| P_REXW, data_reg, TCG_REG_RAX);
		572	+ break;
		573	+ case 2 \| 4:
		574	+ /* movslq */
		575	+ tcg_out_modrm(s, 0x63 \| P_REXW, data_reg, TCG_REG_RAX);
		576	+ break;
		577	+ case 0:
		578	+ case 1:
		579	+ case 2:
		580	+ default:
		581	+ /* movl */
		582	+ tcg_out_modrm(s, 0x8b, data_reg, TCG_REG_RAX);
		583	+ break;
		584	+ case 3:
		585	+ tcg_out_mov(s, data_reg, TCG_REG_RAX);
		586	+ break;
		587	+ }
		588	+
		589	+ /* jmp label2 */
		590	+ tcg_out8(s, 0xeb);
		591	+ label2_ptr = s->code_ptr;
		592	+ s->code_ptr++;
		593	+
		594	+ /* label1: */
		595	+ *label1_ptr = s->code_ptr - label1_ptr - 1;
		596	+
		597	+ /* add x(r1), r0 */
		598	+ tcg_out_modrm_offset(s, 0x03 \| P_REXW, r0, r1, offsetof(CPUTLBEntry, addend) -
		599	+ offsetof(CPUTLBEntry, addr_read));
		600	+#else
		601	+ r0 = addr_reg;
		602	+#endif
		603	+
		604	+#ifdef TARGET_WORDS_BIGENDIAN
		605	+ bswap = 1;
		606	+#else
		607	+ bswap = 0;
		608	+#endif
		609	+ switch(opc) {
		610	+ case 0:
		611	+ /* movzbl */
		612	+ tcg_out_modrm_offset(s, 0xb6 \| P_EXT, data_reg, r0, 0);
		613	+ break;
		614	+ case 0 \| 4:
		615	+ /* movsbX */
		616	+ tcg_out_modrm_offset(s, 0xbe \| P_EXT \| rexw, data_reg, r0, 0);
		617	+ break;
		618	+ case 1:
		619	+ /* movzwl */
		620	+ tcg_out_modrm_offset(s, 0xb7 \| P_EXT, data_reg, r0, 0);
		621	+ if (bswap) {
		622	+ /* rolw $8, data_reg */
		623	+ tcg_out8(s, 0x66);
		624	+ tcg_out_modrm(s, 0xc1, 0, data_reg);
		625	+ tcg_out8(s, 8);
		626	+ }
		627	+ break;
		628	+ case 1 \| 4:
		629	+ if (bswap) {
		630	+ /* movzwl */
		631	+ tcg_out_modrm_offset(s, 0xb7 \| P_EXT, data_reg, r0, 0);
		632	+ /* rolw $8, data_reg */
		633	+ tcg_out8(s, 0x66);
		634	+ tcg_out_modrm(s, 0xc1, 0, data_reg);
		635	+ tcg_out8(s, 8);
		636	+
		637	+ /* movswX data_reg, data_reg */
		638	+ tcg_out_modrm(s, 0xbf \| P_EXT \| rexw, data_reg, data_reg);
		639	+ } else {
		640	+ /* movswX */
		641	+ tcg_out_modrm_offset(s, 0xbf \| P_EXT \| rexw, data_reg, r0, 0);
		642	+ }
		643	+ break;
		644	+ case 2:
		645	+ /* movl (r0), data_reg */
		646	+ tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 0);
		647	+ if (bswap) {
		648	+ /* bswap */
		649	+ tcg_out_opc(s, (0xc8 + (data_reg & 7)) \| P_EXT, 0, data_reg, 0);
		650	+ }
		651	+ break;
		652	+ case 2 \| 4:
		653	+ if (bswap) {
		654	+ /* movl (r0), data_reg */
		655	+ tcg_out_modrm_offset(s, 0x8b, data_reg, r0, 0);
		656	+ /* bswap */
		657	+ tcg_out_opc(s, (0xc8 + (data_reg & 7)) \| P_EXT, 0, data_reg, 0);
		658	+ /* movslq */
		659	+ tcg_out_modrm(s, 0x63 \| P_REXW, data_reg, data_reg);
		660	+ } else {
		661	+ /* movslq */
		662	+ tcg_out_modrm_offset(s, 0x63 \| P_REXW, data_reg, r0, 0);
		663	+ }
		664	+ break;
		665	+ case 3:
		666	+ /* movq (r0), data_reg */
		667	+ tcg_out_modrm_offset(s, 0x8b \| P_REXW, data_reg, r0, 0);
		668	+ if (bswap) {
		669	+ /* bswap */
		670	+ tcg_out_opc(s, (0xc8 + (data_reg & 7)) \| P_EXT \| P_REXW, 0, data_reg, 0);
		671	+ }
		672	+ break;
		673	+ default:
		674	+ tcg_abort();
		675	+ }
		676	+
		677	+#if defined(CONFIG_SOFTMMU)
		678	+ /* label2: */
		679	+ *label2_ptr = s->code_ptr - label2_ptr - 1;
		680	+#endif
		681	+}
		682	+
		683	+static void tcg_out_qemu_st(TCGContext s, const TCGArg args,
		684	+ int opc)
		685	+{
		686	+ int addr_reg, data_reg, r0, r1, mem_index, s_bits, bswap, rexw;
		687	+#if defined(CONFIG_SOFTMMU)
		688	+ uint8_t label1_ptr, label2_ptr;
		689	+#endif
		690	+
		691	+ data_reg = *args++;
		692	+ addr_reg = *args++;
		693	+ mem_index = *args;
		694	+
		695	+ s_bits = opc;
		696	+
		697	+ r0 = TCG_REG_RDI;
		698	+ r1 = TCG_REG_RSI;
		699	+
		700	+#if TARGET_LONG_BITS == 32
		701	+ rexw = 0;
		702	+#else
		703	+ rexw = P_REXW;
		704	+#endif
		705	+#if defined(CONFIG_SOFTMMU)
		706	+ /* mov */
		707	+ tcg_out_modrm(s, 0x8b \| rexw, r1, addr_reg);
		708	+
		709	+ /* mov */
		710	+ tcg_out_modrm(s, 0x8b \| rexw, r0, addr_reg);
		711	+
		712	+ tcg_out_modrm(s, 0xc1 \| rexw, 5, r1); /* shr $x, r1 */
		713	+ tcg_out8(s, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS);
		714	+
		715	+ tcg_out_modrm(s, 0x81 \| rexw, 4, r0); /* andl $x, r0 */
		716	+ tcg_out32(s, TARGET_PAGE_MASK \| ((1 << s_bits) - 1));
		717	+
		718	+ tcg_out_modrm(s, 0x81, 4, r1); /* andl $x, r1 */
		719	+ tcg_out32(s, (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS);
		720	+
		721	+ /* lea offset(r1, env), r1 */
		722	+ tcg_out_modrm_offset2(s, 0x8d \| P_REXW, r1, r1, TCG_AREG0, 0,
		723	+ offsetof(CPUState, tlb_table[mem_index][0].addr_write));
		724	+
		725	+ /* cmp 0(r1), r0 */
		726	+ tcg_out_modrm_offset(s, 0x3b \| rexw, r0, r1, 0);
		727	+
		728	+ /* mov */
		729	+ tcg_out_modrm(s, 0x8b \| rexw, r0, addr_reg);
		730	+
		731	+ /* je label1 */
		732	+ tcg_out8(s, 0x70 + JCC_JE);
		733	+ label1_ptr = s->code_ptr;
		734	+ s->code_ptr++;
		735	+
		736	+ /* XXX: move that code at the end of the TB */
		737	+ switch(opc) {
		738	+ case 0:
		739	+ /* movzbl */
		740	+ tcg_out_modrm(s, 0xb6 \| P_EXT, TCG_REG_RSI, data_reg);
		741	+ break;
		742	+ case 1:
		743	+ /* movzwl */
		744	+ tcg_out_modrm(s, 0xb7 \| P_EXT, TCG_REG_RSI, data_reg);
		745	+ break;
		746	+ case 2:
		747	+ /* movl */
		748	+ tcg_out_modrm(s, 0x8b, TCG_REG_RSI, data_reg);
		749	+ break;
		750	+ default:
		751	+ case 3:
		752	+ tcg_out_mov(s, TCG_REG_RSI, data_reg);
		753	+ break;
		754	+ }
		755	+ tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_RDX, mem_index);
		756	+ tcg_out8(s, 0xe8);
		757	+ tcg_out32(s, (tcg_target_long)qemu_st_helpers[s_bits] -
		758	+ (tcg_target_long)s->code_ptr - 4);
		759	+
		760	+ /* jmp label2 */
		761	+ tcg_out8(s, 0xeb);
		762	+ label2_ptr = s->code_ptr;
		763	+ s->code_ptr++;
		764	+
		765	+ /* label1: */
		766	+ *label1_ptr = s->code_ptr - label1_ptr - 1;
		767	+
		768	+ /* add x(r1), r0 */
		769	+ tcg_out_modrm_offset(s, 0x03 \| P_REXW, r0, r1, offsetof(CPUTLBEntry, addend) -
		770	+ offsetof(CPUTLBEntry, addr_write));
		771	+#else
		772	+ r0 = addr_reg;
		773	+#endif
		774	+
		775	+#ifdef TARGET_WORDS_BIGENDIAN
		776	+ bswap = 1;
		777	+#else
		778	+ bswap = 0;
		779	+#endif
		780	+ switch(opc) {
		781	+ case 0:
		782	+ /* movb */
		783	+ tcg_out_modrm_offset(s, 0x88 \| P_REX, data_reg, r0, 0);
		784	+ break;
		785	+ case 1:
		786	+ if (bswap) {
		787	+ tcg_out_modrm(s, 0x8b, r1, data_reg); /* movl */
		788	+ tcg_out8(s, 0x66); /* rolw $8, %ecx */
		789	+ tcg_out_modrm(s, 0xc1, 0, r1);
		790	+ tcg_out8(s, 8);
		791	+ data_reg = r1;
		792	+ }
		793	+ /* movw */
		794	+ tcg_out8(s, 0x66);
		795	+ tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);
		796	+ break;
		797	+ case 2:
		798	+ if (bswap) {
		799	+ tcg_out_modrm(s, 0x8b, r1, data_reg); /* movl */
		800	+ /* bswap data_reg */
		801	+ tcg_out_opc(s, (0xc8 + r1) \| P_EXT, 0, r1, 0);
		802	+ data_reg = r1;
		803	+ }
		804	+ /* movl */
		805	+ tcg_out_modrm_offset(s, 0x89, data_reg, r0, 0);
		806	+ break;
		807	+ case 3:
		808	+ if (bswap) {
		809	+ tcg_out_mov(s, r1, data_reg);
		810	+ /* bswap data_reg */
		811	+ tcg_out_opc(s, (0xc8 + r1) \| P_EXT \| P_REXW, 0, r1, 0);
		812	+ data_reg = r1;
		813	+ }
		814	+ /* movq */
		815	+ tcg_out_modrm_offset(s, 0x89 \| P_REXW, data_reg, r0, 0);
		816	+ break;
		817	+ default:
		818	+ tcg_abort();
		819	+ }
		820	+
		821	+#if defined(CONFIG_SOFTMMU)
		822	+ /* label2: */
		823	+ *label2_ptr = s->code_ptr - label2_ptr - 1;
		824	+#endif
		825	+}
		826	+
		827	+static inline void tcg_out_op(TCGContext s, int opc, const TCGArg args,
		828	+ const int *const_args)
		829	+{
		830	+ int c;
		831	+
		832	+ switch(opc) {
		833	+ case INDEX_op_exit_tb:
		834	+ tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_RAX, args[0]);
		835	+ tcg_out8(s, 0xc3); /* ret */
		836	+ break;
		837	+ case INDEX_op_goto_tb:
		838	+ if (s->tb_jmp_offset) {
		839	+ /* direct jump method */
		840	+ tcg_out8(s, 0xe9); /* jmp im */
		841	+ s->tb_jmp_offset[args[0]] = s->code_ptr - s->code_buf;
		842	+ tcg_out32(s, 0);
		843	+ } else {
		844	+ /* indirect jump method */
		845	+ /* jmp Ev */
		846	+ tcg_out_modrm_offset(s, 0xff, 4, -1,
		847	+ (tcg_target_long)(s->tb_next +
		848	+ args[0]));
		849	+ }
		850	+ s->tb_next_offset[args[0]] = s->code_ptr - s->code_buf;
		851	+ break;
		852	+ case INDEX_op_call:
		853	+ if (const_args[0]) {
		854	+ tcg_out8(s, 0xe8);
		855	+ tcg_out32(s, args[0] - (tcg_target_long)s->code_ptr - 4);
		856	+ } else {
		857	+ tcg_out_modrm(s, 0xff, 2, args[0]);
		858	+ }
		859	+ break;
		860	+ case INDEX_op_jmp:
		861	+ if (const_args[0]) {
		862	+ tcg_out8(s, 0xe9);
		863	+ tcg_out32(s, args[0] - (tcg_target_long)s->code_ptr - 4);
		864	+ } else {
		865	+ tcg_out_modrm(s, 0xff, 4, args[0]);
		866	+ }
		867	+ break;
		868	+ case INDEX_op_br:
		869	+ tcg_out_jxx(s, JCC_JMP, args[0]);
		870	+ break;
		871	+ case INDEX_op_movi_i32:
		872	+ tcg_out_movi(s, TCG_TYPE_I32, args[0], (uint32_t)args[1]);
		873	+ break;
		874	+ case INDEX_op_movi_i64:
		875	+ tcg_out_movi(s, TCG_TYPE_I64, args[0], args[1]);
		876	+ break;
		877	+ case INDEX_op_ld8u_i32:
		878	+ case INDEX_op_ld8u_i64:
		879	+ /* movzbl */
		880	+ tcg_out_modrm_offset(s, 0xb6 \| P_EXT, args[0], args[1], args[2]);
		881	+ break;
		882	+ case INDEX_op_ld8s_i32:
		883	+ /* movsbl */
		884	+ tcg_out_modrm_offset(s, 0xbe \| P_EXT, args[0], args[1], args[2]);
		885	+ break;
		886	+ case INDEX_op_ld8s_i64:
		887	+ /* movsbq */
		888	+ tcg_out_modrm_offset(s, 0xbe \| P_EXT \| P_REXW, args[0], args[1], args[2]);
		889	+ break;
		890	+ case INDEX_op_ld16u_i32:
		891	+ case INDEX_op_ld16u_i64:
		892	+ /* movzwl */
		893	+ tcg_out_modrm_offset(s, 0xb7 \| P_EXT, args[0], args[1], args[2]);
		894	+ break;
		895	+ case INDEX_op_ld16s_i32:
		896	+ /* movswl */
		897	+ tcg_out_modrm_offset(s, 0xbf \| P_EXT, args[0], args[1], args[2]);
		898	+ break;
		899	+ case INDEX_op_ld16s_i64:
		900	+ /* movswq */
		901	+ tcg_out_modrm_offset(s, 0xbf \| P_EXT \| P_REXW, args[0], args[1], args[2]);
		902	+ break;
		903	+ case INDEX_op_ld_i32:
		904	+ case INDEX_op_ld32u_i64:
		905	+ /* movl */
		906	+ tcg_out_modrm_offset(s, 0x8b, args[0], args[1], args[2]);
		907	+ break;
		908	+ case INDEX_op_ld32s_i64:
		909	+ /* movslq */
		910	+ tcg_out_modrm_offset(s, 0x63 \| P_REXW, args[0], args[1], args[2]);
		911	+ break;
		912	+ case INDEX_op_ld_i64:
		913	+ /* movq */
		914	+ tcg_out_modrm_offset(s, 0x8b \| P_REXW, args[0], args[1], args[2]);
		915	+ break;
		916	+
		917	+ case INDEX_op_st8_i32:
		918	+ case INDEX_op_st8_i64:
		919	+ /* movb */
		920	+ tcg_out_modrm_offset(s, 0x88 \| P_REX, args[0], args[1], args[2]);
		921	+ break;
		922	+ case INDEX_op_st16_i32:
		923	+ case INDEX_op_st16_i64:
		924	+ /* movw */
		925	+ tcg_out8(s, 0x66);
		926	+ tcg_out_modrm_offset(s, 0x89, args[0], args[1], args[2]);
		927	+ break;
		928	+ case INDEX_op_st_i32:
		929	+ case INDEX_op_st32_i64:
		930	+ /* movl */
		931	+ tcg_out_modrm_offset(s, 0x89, args[0], args[1], args[2]);
		932	+ break;
		933	+ case INDEX_op_st_i64:
		934	+ /* movq */
		935	+ tcg_out_modrm_offset(s, 0x89 \| P_REXW, args[0], args[1], args[2]);
		936	+ break;
		937	+
		938	+ case INDEX_op_sub_i32:
		939	+ c = ARITH_SUB;
		940	+ goto gen_arith32;
		941	+ case INDEX_op_and_i32:
		942	+ c = ARITH_AND;
		943	+ goto gen_arith32;
		944	+ case INDEX_op_or_i32:
		945	+ c = ARITH_OR;
		946	+ goto gen_arith32;
		947	+ case INDEX_op_xor_i32:
		948	+ c = ARITH_XOR;
		949	+ goto gen_arith32;
		950	+ case INDEX_op_add_i32:
		951	+ c = ARITH_ADD;
		952	+ gen_arith32:
		953	+ if (const_args[2]) {
		954	+ tgen_arithi32(s, c, args[0], args[2]);
		955	+ } else {
		956	+ tcg_out_modrm(s, 0x01 \| (c << 3), args[2], args[0]);
		957	+ }
		958	+ break;
		959	+
		960	+ case INDEX_op_sub_i64:
		961	+ c = ARITH_SUB;
		962	+ goto gen_arith64;
		963	+ case INDEX_op_and_i64:
		964	+ c = ARITH_AND;
		965	+ goto gen_arith64;
		966	+ case INDEX_op_or_i64:
		967	+ c = ARITH_OR;
		968	+ goto gen_arith64;
		969	+ case INDEX_op_xor_i64:
		970	+ c = ARITH_XOR;
		971	+ goto gen_arith64;
		972	+ case INDEX_op_add_i64:
		973	+ c = ARITH_ADD;
		974	+ gen_arith64:
		975	+ if (const_args[2]) {
		976	+ tgen_arithi64(s, c, args[0], args[2]);
		977	+ } else {
		978	+ tcg_out_modrm(s, 0x01 \| (c << 3) \| P_REXW, args[2], args[0]);
		979	+ }
		980	+ break;
		981	+
		982	+ case INDEX_op_mul_i32:
		983	+ if (const_args[2]) {
		984	+ int32_t val;
		985	+ val = args[2];
		986	+ if (val == (int8_t)val) {
		987	+ tcg_out_modrm(s, 0x6b, args[0], args[0]);
		988	+ tcg_out8(s, val);
		989	+ } else {
		990	+ tcg_out_modrm(s, 0x69, args[0], args[0]);
		991	+ tcg_out32(s, val);
		992	+ }
		993	+ } else {
		994	+ tcg_out_modrm(s, 0xaf \| P_EXT, args[0], args[2]);
		995	+ }
		996	+ break;
		997	+ case INDEX_op_mul_i64:
		998	+ if (const_args[2]) {
		999	+ int32_t val;
		1000	+ val = args[2];
		1001	+ if (val == (int8_t)val) {
		1002	+ tcg_out_modrm(s, 0x6b \| P_REXW, args[0], args[0]);
		1003	+ tcg_out8(s, val);
		1004	+ } else {
		1005	+ tcg_out_modrm(s, 0x69 \| P_REXW, args[0], args[0]);
		1006	+ tcg_out32(s, val);
		1007	+ }
		1008	+ } else {
		1009	+ tcg_out_modrm(s, 0xaf \| P_EXT \| P_REXW, args[0], args[2]);
		1010	+ }
		1011	+ break;
		1012	+ case INDEX_op_div2_i32:
		1013	+ tcg_out_modrm(s, 0xf7, 7, args[4]);
		1014	+ break;
		1015	+ case INDEX_op_divu2_i32:
		1016	+ tcg_out_modrm(s, 0xf7, 6, args[4]);
		1017	+ break;
		1018	+ case INDEX_op_div2_i64:
		1019	+ tcg_out_modrm(s, 0xf7 \| P_REXW, 7, args[4]);
		1020	+ break;
		1021	+ case INDEX_op_divu2_i64:
		1022	+ tcg_out_modrm(s, 0xf7 \| P_REXW, 6, args[4]);
		1023	+ break;
		1024	+
		1025	+ case INDEX_op_shl_i32:
		1026	+ c = SHIFT_SHL;
		1027	+ gen_shift32:
		1028	+ if (const_args[2]) {
		1029	+ if (args[2] == 1) {
		1030	+ tcg_out_modrm(s, 0xd1, c, args[0]);
		1031	+ } else {
		1032	+ tcg_out_modrm(s, 0xc1, c, args[0]);
		1033	+ tcg_out8(s, args[2]);
		1034	+ }
		1035	+ } else {
		1036	+ tcg_out_modrm(s, 0xd3, c, args[0]);
		1037	+ }
		1038	+ break;
		1039	+ case INDEX_op_shr_i32:
		1040	+ c = SHIFT_SHR;
		1041	+ goto gen_shift32;
		1042	+ case INDEX_op_sar_i32:
		1043	+ c = SHIFT_SAR;
		1044	+ goto gen_shift32;
		1045	+
		1046	+ case INDEX_op_shl_i64:
		1047	+ c = SHIFT_SHL;
		1048	+ gen_shift64:
		1049	+ if (const_args[2]) {
		1050	+ if (args[2] == 1) {
		1051	+ tcg_out_modrm(s, 0xd1 \| P_REXW, c, args[0]);
		1052	+ } else {
		1053	+ tcg_out_modrm(s, 0xc1 \| P_REXW, c, args[0]);
		1054	+ tcg_out8(s, args[2]);
		1055	+ }
		1056	+ } else {
		1057	+ tcg_out_modrm(s, 0xd3 \| P_REXW, c, args[0]);
		1058	+ }
		1059	+ break;
		1060	+ case INDEX_op_shr_i64:
		1061	+ c = SHIFT_SHR;
		1062	+ goto gen_shift64;
		1063	+ case INDEX_op_sar_i64:
		1064	+ c = SHIFT_SAR;
		1065	+ goto gen_shift64;
		1066	+
		1067	+ case INDEX_op_brcond_i32:
		1068	+ tcg_out_brcond(s, args[2], args[0], args[1], const_args[1],
		1069	+ args[3], 0);
		1070	+ break;
		1071	+ case INDEX_op_brcond_i64:
		1072	+ tcg_out_brcond(s, args[2], args[0], args[1], const_args[1],
		1073	+ args[3], P_REXW);
		1074	+ break;
		1075	+
		1076	+ case INDEX_op_bswap_i32:
		1077	+ tcg_out_opc(s, (0xc8 + (args[0] & 7)) \| P_EXT, 0, args[0], 0);
		1078	+ break;
		1079	+ case INDEX_op_bswap_i64:
		1080	+ tcg_out_opc(s, (0xc8 + (args[0] & 7)) \| P_EXT \| P_REXW, 0, args[0], 0);
		1081	+ break;
		1082	+
		1083	+ case INDEX_op_qemu_ld8u:
		1084	+ tcg_out_qemu_ld(s, args, 0);
		1085	+ break;
		1086	+ case INDEX_op_qemu_ld8s:
		1087	+ tcg_out_qemu_ld(s, args, 0 \| 4);
		1088	+ break;
		1089	+ case INDEX_op_qemu_ld16u:
		1090	+ tcg_out_qemu_ld(s, args, 1);
		1091	+ break;
		1092	+ case INDEX_op_qemu_ld16s:
		1093	+ tcg_out_qemu_ld(s, args, 1 \| 4);
		1094	+ break;
		1095	+ case INDEX_op_qemu_ld32u:
		1096	+ tcg_out_qemu_ld(s, args, 2);
		1097	+ break;
		1098	+ case INDEX_op_qemu_ld32s:
		1099	+ tcg_out_qemu_ld(s, args, 2 \| 4);
		1100	+ break;
		1101	+ case INDEX_op_qemu_ld64:
		1102	+ tcg_out_qemu_ld(s, args, 3);
		1103	+ break;
		1104	+
		1105	+ case INDEX_op_qemu_st8:
		1106	+ tcg_out_qemu_st(s, args, 0);
		1107	+ break;
		1108	+ case INDEX_op_qemu_st16:
		1109	+ tcg_out_qemu_st(s, args, 1);
		1110	+ break;
		1111	+ case INDEX_op_qemu_st32:
		1112	+ tcg_out_qemu_st(s, args, 2);
		1113	+ break;
		1114	+ case INDEX_op_qemu_st64:
		1115	+ tcg_out_qemu_st(s, args, 3);
		1116	+ break;
		1117	+
		1118	+ default:
		1119	+ tcg_abort();
		1120	+ }
		1121	+}
		1122	+
		1123	+static const TCGTargetOpDef x86_64_op_defs[] = {
		1124	+ { INDEX_op_exit_tb, { } },
		1125	+ { INDEX_op_goto_tb, { } },
		1126	+ { INDEX_op_call, { "ri" } }, /* XXX: might need a specific constant constraint */
		1127	+ { INDEX_op_jmp, { "ri" } }, /* XXX: might need a specific constant constraint */
		1128	+ { INDEX_op_br, { } },
		1129	+
		1130	+ { INDEX_op_mov_i32, { "r", "r" } },
		1131	+ { INDEX_op_movi_i32, { "r" } },
		1132	+ { INDEX_op_ld8u_i32, { "r", "r" } },
		1133	+ { INDEX_op_ld8s_i32, { "r", "r" } },
		1134	+ { INDEX_op_ld16u_i32, { "r", "r" } },
		1135	+ { INDEX_op_ld16s_i32, { "r", "r" } },
		1136	+ { INDEX_op_ld_i32, { "r", "r" } },
		1137	+ { INDEX_op_st8_i32, { "r", "r" } },
		1138	+ { INDEX_op_st16_i32, { "r", "r" } },
		1139	+ { INDEX_op_st_i32, { "r", "r" } },
		1140	+
		1141	+ { INDEX_op_add_i32, { "r", "0", "ri" } },
		1142	+ { INDEX_op_mul_i32, { "r", "0", "ri" } },
		1143	+ { INDEX_op_div2_i32, { "a", "d", "0", "1", "r" } },
		1144	+ { INDEX_op_divu2_i32, { "a", "d", "0", "1", "r" } },
		1145	+ { INDEX_op_sub_i32, { "r", "0", "ri" } },
		1146	+ { INDEX_op_and_i32, { "r", "0", "ri" } },
		1147	+ { INDEX_op_or_i32, { "r", "0", "ri" } },
		1148	+ { INDEX_op_xor_i32, { "r", "0", "ri" } },
		1149	+
		1150	+ { INDEX_op_shl_i32, { "r", "0", "ci" } },
		1151	+ { INDEX_op_shr_i32, { "r", "0", "ci" } },
		1152	+ { INDEX_op_sar_i32, { "r", "0", "ci" } },
		1153	+
		1154	+ { INDEX_op_brcond_i32, { "r", "ri" } },
		1155	+
		1156	+ { INDEX_op_mov_i64, { "r", "r" } },
		1157	+ { INDEX_op_movi_i64, { "r" } },
		1158	+ { INDEX_op_ld8u_i64, { "r", "r" } },
		1159	+ { INDEX_op_ld8s_i64, { "r", "r" } },
		1160	+ { INDEX_op_ld16u_i64, { "r", "r" } },
		1161	+ { INDEX_op_ld16s_i64, { "r", "r" } },
		1162	+ { INDEX_op_ld32u_i64, { "r", "r" } },
		1163	+ { INDEX_op_ld32s_i64, { "r", "r" } },
		1164	+ { INDEX_op_ld_i64, { "r", "r" } },
		1165	+ { INDEX_op_st8_i64, { "r", "r" } },
		1166	+ { INDEX_op_st16_i64, { "r", "r" } },
		1167	+ { INDEX_op_st32_i64, { "r", "r" } },
		1168	+ { INDEX_op_st_i64, { "r", "r" } },
		1169	+
		1170	+ { INDEX_op_add_i64, { "r", "0", "re" } },
		1171	+ { INDEX_op_mul_i64, { "r", "0", "re" } },
		1172	+ { INDEX_op_div2_i64, { "a", "d", "0", "1", "r" } },
		1173	+ { INDEX_op_divu2_i64, { "a", "d", "0", "1", "r" } },
		1174	+ { INDEX_op_sub_i64, { "r", "0", "re" } },
		1175	+ { INDEX_op_and_i64, { "r", "0", "reZ" } },
		1176	+ { INDEX_op_or_i64, { "r", "0", "re" } },
		1177	+ { INDEX_op_xor_i64, { "r", "0", "re" } },
		1178	+
		1179	+ { INDEX_op_shl_i64, { "r", "0", "ci" } },
		1180	+ { INDEX_op_shr_i64, { "r", "0", "ci" } },
		1181	+ { INDEX_op_sar_i64, { "r", "0", "ci" } },
		1182	+
		1183	+ { INDEX_op_brcond_i64, { "r", "re" } },
		1184	+
		1185	+ { INDEX_op_bswap_i32, { "r", "0" } },
		1186	+ { INDEX_op_bswap_i64, { "r", "0" } },
		1187	+
		1188	+ { INDEX_op_qemu_ld8u, { "r", "L" } },
		1189	+ { INDEX_op_qemu_ld8s, { "r", "L" } },
		1190	+ { INDEX_op_qemu_ld16u, { "r", "L" } },
		1191	+ { INDEX_op_qemu_ld16s, { "r", "L" } },
		1192	+ { INDEX_op_qemu_ld32u, { "r", "L" } },
		1193	+ { INDEX_op_qemu_ld32s, { "r", "L" } },
		1194	+ { INDEX_op_qemu_ld64, { "r", "L" } },
		1195	+
		1196	+ { INDEX_op_qemu_st8, { "L", "L" } },
		1197	+ { INDEX_op_qemu_st16, { "L", "L" } },
		1198	+ { INDEX_op_qemu_st32, { "L", "L" } },
		1199	+ { INDEX_op_qemu_st64, { "L", "L", "L" } },
		1200	+
		1201	+ { -1 },
		1202	+};
		1203	+
		1204	+void tcg_target_init(TCGContext *s)
		1205	+{
		1206	+ tcg_regset_set32(tcg_target_available_regs[TCG_TYPE_I32], 0, 0xffff);
		1207	+ tcg_regset_set32(tcg_target_available_regs[TCG_TYPE_I64], 0, 0xffff);
		1208	+ tcg_regset_set32(tcg_target_call_clobber_regs, 0,
		1209	+ (1 << TCG_REG_RDI) \|
		1210	+ (1 << TCG_REG_RSI) \|
		1211	+ (1 << TCG_REG_RDX) \|
		1212	+ (1 << TCG_REG_RCX) \|
		1213	+ (1 << TCG_REG_R8) \|
		1214	+ (1 << TCG_REG_R9) \|
		1215	+ (1 << TCG_REG_RAX) \|
		1216	+ (1 << TCG_REG_R10) \|
		1217	+ (1 << TCG_REG_R11));
		1218	+
		1219	+ tcg_regset_clear(s->reserved_regs);
		1220	+ tcg_regset_set_reg(s->reserved_regs, TCG_REG_RSP);
		1221	+ /* XXX: will be suppresed when proper global TB entry code will be
		1222	+ generated */
		1223	+ tcg_regset_set_reg(s->reserved_regs, TCG_REG_RBX);
		1224	+ tcg_regset_set_reg(s->reserved_regs, TCG_REG_RBP);
		1225	+
		1226	+ tcg_add_target_add_op_defs(x86_64_op_defs);
		1227	+}

tcg/x86_64/tcg-target.h 0 → 100644

View file @c896fe2

		1	+/*
		2	+ * Tiny Code Generator for QEMU
		3	+ *
		4	+ * Copyright (c) 2008 Fabrice Bellard
		5	+ *
		6	+ * Permission is hereby granted, free of charge, to any person obtaining a copy
		7	+ * of this software and associated documentation files (the "Software"), to deal
		8	+ * in the Software without restriction, including without limitation the rights
		9	+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
		10	+ * copies of the Software, and to permit persons to whom the Software is
		11	+ * furnished to do so, subject to the following conditions:
		12	+ *
		13	+ * The above copyright notice and this permission notice shall be included in
		14	+ * all copies or substantial portions of the Software.
		15	+ *
		16	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
		17	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
		18	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
		19	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
		20	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
		21	+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
		22	+ * THE SOFTWARE.
		23	+ */
		24	+#define TCG_TARGET_X86_64 1
		25	+
		26	+#define TCG_TARGET_REG_BITS 64
		27	+//#define TCG_TARGET_WORDS_BIGENDIAN
		28	+
		29	+#define TCG_TARGET_NB_REGS 16
		30	+
		31	+enum {
		32	+ TCG_REG_RAX = 0,
		33	+ TCG_REG_RCX,
		34	+ TCG_REG_RDX,
		35	+ TCG_REG_RBX,
		36	+ TCG_REG_RSP,
		37	+ TCG_REG_RBP,
		38	+ TCG_REG_RSI,
		39	+ TCG_REG_RDI,
		40	+ TCG_REG_R8,
		41	+ TCG_REG_R9,
		42	+ TCG_REG_R10,
		43	+ TCG_REG_R11,
		44	+ TCG_REG_R12,
		45	+ TCG_REG_R13,
		46	+ TCG_REG_R14,
		47	+ TCG_REG_R15,
		48	+};
		49	+
		50	+#define TCG_CT_CONST_S32 0x100
		51	+#define TCG_CT_CONST_U32 0x200
		52	+
		53	+/* used for function call generation */
		54	+#define TCG_REG_CALL_STACK TCG_REG_RSP
		55	+#define TCG_TARGET_STACK_ALIGN 16
		56	+
		57	+/* optional instructions */
		58	+#define TCG_TARGET_HAS_bswap_i32
		59	+#define TCG_TARGET_HAS_bswap_i64
		60	+
		61	+/* Note: must be synced with dyngen-exec.h */
		62	+#define TCG_AREG0 TCG_REG_R14
		63	+#define TCG_AREG1 TCG_REG_R15
		64	+#define TCG_AREG2 TCG_REG_R12
		65	+#define TCG_AREG3 TCG_REG_R13
		66	+
		67	+static inline void flush_icache_range(unsigned long start, unsigned long stop)
		68	+{
		69	+}

gwj / at91sam9263 · Commits

GitLab

TCG code generator